├── .gitignore ├── CHANGELOG ├── LICENSE ├── README.md ├── TODO ├── data └── exaddos │ └── html │ ├── exa.jpg │ ├── favicon.ico │ ├── inc │ ├── jquery-1.10.2.js │ ├── jquery.tablesorter.js │ └── jquery.tmpl.min.js │ ├── index.html │ ├── overview.html │ ├── talker.html │ └── talkers.html ├── etc └── exaddos │ ├── exaddos.conf.snmpv2 │ └── exaddos.conf.snmpv3 ├── lib └── exaddos │ ├── __init__.py │ ├── application.py │ ├── configuration.py │ ├── container.py │ ├── debug.py │ ├── flow.py │ ├── http.py │ ├── ipfix.py │ ├── leak │ ├── __init__.py │ ├── gcdump.py │ └── objgraph.py │ ├── log.py │ ├── q.py │ ├── reactor.py │ ├── snmp.py │ ├── thread.py │ └── warning.py ├── sbin └── exaddos └── test ├── .empty └── ipfix.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Private configuration files 2 | *.exa 3 | 4 | # Python 5 | *.py[cod] 6 | 7 | # C extensions 8 | *.so 9 | 10 | # Packages 11 | *.egg 12 | *.egg-info 13 | dist 14 | build 15 | eggs 16 | parts 17 | bin 18 | var 19 | sdist 20 | develop-eggs 21 | .installed.cfg 22 | lib64 23 | __pycache__ 24 | 25 | # Installer logs 26 | pip-log.txt 27 | 28 | # Unit test / coverage reports 29 | .coverage 30 | .tox 31 | nosetests.xml 32 | 33 | # Translations 34 | *.mo 35 | 36 | # Mr Developer 37 | .mr.developer.cfg 38 | .project 39 | .pydevproject 40 | -------------------------------------------------------------------------------- /CHANGELOG: -------------------------------------------------------------------------------- 1 | Version 0.5.0 2 | * better format of units in overview 3 | 4 | Version 0.4.0 5 | * Save PID 6 | 7 | Version 0.3.0 8 | * Per IP monitoring page 9 | * More rendering in JS 10 | * Moving flow parsing to multiple threads 11 | * Support for daemonize, better logging 12 | 13 | Version 0.2.0 14 | * Src/Dst port top talkers 15 | * better / less json calls 16 | * SNMPv3 support 17 | requested by: Ryan Steinmetz 18 | 19 | Version 0.1.0 20 | * Initial release 21 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2014, Exa Networks 2 | Copyright (c) 2014, Thomas Mangin 3 | Copyright (c) 2014, Daniel Piekacz 4 | Copyright (c) 2014, Michael Lancaster 5 | 6 | All rights reserved. 7 | 8 | Redistribution and use in source and binary forms, with or without modification, 9 | are permitted provided that the following conditions are met: 10 | 11 | * Redistributions of source code must retain the above copyright notice, this 12 | list of conditions and the following disclaimer. 13 | 14 | * Redistributions in binary form must reproduce the above copyright notice, this 15 | list of conditions and the following disclaimer in the documentation and/or 16 | other materials provided with the distribution. 17 | 18 | * Neither the name of Exa Networks Limited nor the names of its 19 | contributors may be used to endorse or promote products derived from 20 | this software without specific prior written permission. 21 | 22 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 23 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 24 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 25 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR 26 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 27 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 28 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 29 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 30 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 31 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 32 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | exaddos 2 | ======= 3 | 4 | Monitor your network for DDOS 5 | 6 | ExaDDOS is an application able to gather different data sources to present a real time unified view of your network. 7 | 8 | It can gather : 9 | - SNMP information at your edge 10 | - IPFIX export from your routers 11 | 12 | And present it using a web interface. Our goal is to very quickly integrate it with ExaBGP to allow a "one click" anti-DDOS solution. 13 | 14 | The tools is still in development (no release was made yet) and expected to quickly gain features over time. 15 | 16 | It is not designed to replace tools like NSFEN which do record and allow complex search on saved record, but to provide an "in flight" view of the flows currently passing through the network. 17 | 18 | Our current solution includes: 19 | - An RRD based solution for interface traffic graphing 20 | - [AS-STATS](https://neon1.net/as-stats/) to find which peers are our top talkers 21 | - [NFSEN](http://nfsen.sourceforge.net/) to collect, store and search flows 22 | - An ExaDDOS like internal solution, to quickly identify which IPs are causing an attack 23 | 24 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | log better the different thread issues so they can be reported in the interface 2 | return all value as number via JSON and format it in the page 3 | move the inserting in container in its own thread to not miss on UDP packets 4 | reduce the time the lock is taken for the flow container 5 | find the sweet spot for the number of decoding thread per ipfix server thread (the number 3 comes from nowhere) 6 | -------------------------------------------------------------------------------- /data/exaddos/html/exa.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Exa-Networks/exaddos/15c5e0a5575bab4c8fbac31eeb6790eb9b710a39/data/exaddos/html/exa.jpg -------------------------------------------------------------------------------- /data/exaddos/html/favicon.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Exa-Networks/exaddos/15c5e0a5575bab4c8fbac31eeb6790eb9b710a39/data/exaddos/html/favicon.ico -------------------------------------------------------------------------------- /data/exaddos/html/inc/jquery.tablesorter.js: -------------------------------------------------------------------------------- 1 | /* 2 | * 3 | * TableSorter 2.0 - Client-side table sorting with ease! 4 | * Version 2.0.5b 5 | * @requires jQuery v1.2.3 6 | * 7 | * Copyright (c) 2007 Christian Bach 8 | * Examples and docs at: http://tablesorter.com 9 | * Dual licensed under the MIT and GPL licenses: 10 | * http://www.opensource.org/licenses/mit-license.php 11 | * http://www.gnu.org/licenses/gpl.html 12 | * 13 | */ 14 | /** 15 | * 16 | * @description Create a sortable table with multi-column sorting capabilitys 17 | * 18 | * @example $('table').tablesorter(); 19 | * @desc Create a simple tablesorter interface. 20 | * 21 | * @example $('table').tablesorter({ sortList:[[0,0],[1,0]] }); 22 | * @desc Create a tablesorter interface and sort on the first and secound column column headers. 23 | * 24 | * @example $('table').tablesorter({ headers: { 0: { sorter: false}, 1: {sorter: false} } }); 25 | * 26 | * @desc Create a tablesorter interface and disableing the first and second column headers. 27 | * 28 | * 29 | * @example $('table').tablesorter({ headers: { 0: {sorter:"integer"}, 1: {sorter:"currency"} } }); 30 | * 31 | * @desc Create a tablesorter interface and set a column parser for the first 32 | * and second column. 33 | * 34 | * 35 | * @param Object 36 | * settings An object literal containing key/value pairs to provide 37 | * optional settings. 38 | * 39 | * 40 | * @option String cssHeader (optional) A string of the class name to be appended 41 | * to sortable tr elements in the thead of the table. Default value: 42 | * "header" 43 | * 44 | * @option String cssAsc (optional) A string of the class name to be appended to 45 | * sortable tr elements in the thead on a ascending sort. Default value: 46 | * "headerSortUp" 47 | * 48 | * @option String cssDesc (optional) A string of the class name to be appended 49 | * to sortable tr elements in the thead on a descending sort. Default 50 | * value: "headerSortDown" 51 | * 52 | * @option String sortInitialOrder (optional) A string of the inital sorting 53 | * order can be asc or desc. Default value: "asc" 54 | * 55 | * @option String sortMultisortKey (optional) A string of the multi-column sort 56 | * key. Default value: "shiftKey" 57 | * 58 | * @option String textExtraction (optional) A string of the text-extraction 59 | * method to use. For complex html structures inside td cell set this 60 | * option to "complex", on large tables the complex option can be slow. 61 | * Default value: "simple" 62 | * 63 | * @option Object headers (optional) An array containing the forces sorting 64 | * rules. This option let's you specify a default sorting rule. Default 65 | * value: null 66 | * 67 | * @option Array sortList (optional) An array containing the forces sorting 68 | * rules. This option let's you specify a default sorting rule. Default 69 | * value: null 70 | * 71 | * @option Array sortForce (optional) An array containing forced sorting rules. 72 | * This option let's you specify a default sorting rule, which is 73 | * prepended to user-selected rules. Default value: null 74 | * 75 | * @option Boolean sortLocaleCompare (optional) Boolean flag indicating whatever 76 | * to use String.localeCampare method or not. Default set to true. 77 | * 78 | * 79 | * @option Array sortAppend (optional) An array containing forced sorting rules. 80 | * This option let's you specify a default sorting rule, which is 81 | * appended to user-selected rules. Default value: null 82 | * 83 | * @option Boolean widthFixed (optional) Boolean flag indicating if tablesorter 84 | * should apply fixed widths to the table columns. This is usefull when 85 | * using the pager companion plugin. This options requires the dimension 86 | * jquery plugin. Default value: false 87 | * 88 | * @option Boolean cancelSelection (optional) Boolean flag indicating if 89 | * tablesorter should cancel selection of the table headers text. 90 | * Default value: true 91 | * 92 | * @option Boolean debug (optional) Boolean flag indicating if tablesorter 93 | * should display debuging information usefull for development. 94 | * 95 | * @type jQuery 96 | * 97 | * @name tablesorter 98 | * 99 | * @cat Plugins/Tablesorter 100 | * 101 | * @author Christian Bach/christian.bach@polyester.se 102 | */ 103 | 104 | (function ($) { 105 | $.extend({ 106 | tablesorter: new 107 | function () { 108 | 109 | var parsers = [], 110 | widgets = []; 111 | 112 | this.defaults = { 113 | cssHeader: "header", 114 | cssAsc: "headerSortUp", 115 | cssDesc: "headerSortDown", 116 | cssChildRow: "expand-child", 117 | sortInitialOrder: "asc", 118 | sortMultiSortKey: "shiftKey", 119 | sortForce: null, 120 | sortAppend: null, 121 | sortLocaleCompare: true, 122 | textExtraction: "simple", 123 | parsers: {}, widgets: [], 124 | widgetZebra: { 125 | css: ["even", "odd"] 126 | }, headers: {}, widthFixed: false, 127 | cancelSelection: true, 128 | sortList: [], 129 | headerList: [], 130 | dateFormat: "us", 131 | decimal: '/\.|\,/g', 132 | onRenderHeader: null, 133 | selectorHeaders: 'thead th', 134 | debug: false 135 | }; 136 | 137 | /* debuging utils */ 138 | 139 | function benchmark(s, d) { 140 | log(s + "," + (new Date().getTime() - d.getTime()) + "ms"); 141 | } 142 | 143 | this.benchmark = benchmark; 144 | 145 | function log(s) { 146 | if (typeof console != "undefined" && typeof console.debug != "undefined") { 147 | console.log(s); 148 | } else { 149 | alert(s); 150 | } 151 | } 152 | 153 | /* parsers utils */ 154 | 155 | function buildParserCache(table, $headers) { 156 | 157 | if (table.config.debug) { 158 | var parsersDebug = ""; 159 | } 160 | 161 | if (table.tBodies.length == 0) return; // In the case of empty tables 162 | var rows = table.tBodies[0].rows; 163 | 164 | if (rows[0]) { 165 | 166 | var list = [], 167 | cells = rows[0].cells, 168 | l = cells.length; 169 | 170 | for (var i = 0; i < l; i++) { 171 | 172 | var p = false; 173 | 174 | if ($.metadata && ($($headers[i]).metadata() && $($headers[i]).metadata().sorter)) { 175 | 176 | p = getParserById($($headers[i]).metadata().sorter); 177 | 178 | } else if ((table.config.headers[i] && table.config.headers[i].sorter)) { 179 | 180 | p = getParserById(table.config.headers[i].sorter); 181 | } 182 | if (!p) { 183 | 184 | p = detectParserForColumn(table, rows, -1, i); 185 | } 186 | 187 | if (table.config.debug) { 188 | parsersDebug += "column:" + i + " parser:" + p.id + "\n"; 189 | } 190 | 191 | list.push(p); 192 | } 193 | } 194 | 195 | if (table.config.debug) { 196 | log(parsersDebug); 197 | } 198 | 199 | return list; 200 | }; 201 | 202 | function detectParserForColumn(table, rows, rowIndex, cellIndex) { 203 | var l = parsers.length, 204 | node = false, 205 | nodeValue = false, 206 | keepLooking = true; 207 | while (nodeValue == '' && keepLooking) { 208 | rowIndex++; 209 | if (rows[rowIndex]) { 210 | node = getNodeFromRowAndCellIndex(rows, rowIndex, cellIndex); 211 | nodeValue = trimAndGetNodeText(table.config, node); 212 | if (table.config.debug) { 213 | log('Checking if value was empty on row:' + rowIndex); 214 | } 215 | } else { 216 | keepLooking = false; 217 | } 218 | } 219 | for (var i = 1; i < l; i++) { 220 | if (parsers[i].is(nodeValue, table, node)) { 221 | return parsers[i]; 222 | } 223 | } 224 | // 0 is always the generic parser (text) 225 | return parsers[0]; 226 | } 227 | 228 | function getNodeFromRowAndCellIndex(rows, rowIndex, cellIndex) { 229 | return rows[rowIndex].cells[cellIndex]; 230 | } 231 | 232 | function trimAndGetNodeText(config, node) { 233 | return $.trim(getElementText(config, node)); 234 | } 235 | 236 | function getParserById(name) { 237 | var l = parsers.length; 238 | for (var i = 0; i < l; i++) { 239 | if (parsers[i].id.toLowerCase() == name.toLowerCase()) { 240 | return parsers[i]; 241 | } 242 | } 243 | return false; 244 | } 245 | 246 | /* utils */ 247 | 248 | function buildCache(table) { 249 | 250 | if (table.config.debug) { 251 | var cacheTime = new Date(); 252 | } 253 | 254 | var totalRows = (table.tBodies[0] && table.tBodies[0].rows.length) || 0, 255 | totalCells = (table.tBodies[0].rows[0] && table.tBodies[0].rows[0].cells.length) || 0, 256 | parsers = table.config.parsers, 257 | cache = { 258 | row: [], 259 | normalized: [] 260 | }; 261 | 262 | for (var i = 0; i < totalRows; ++i) { 263 | 264 | /** Add the table data to main data array */ 265 | var c = $(table.tBodies[0].rows[i]), 266 | cols = []; 267 | 268 | // if this is a child row, add it to the last row's children and 269 | // continue to the next row 270 | if (c.hasClass(table.config.cssChildRow)) { 271 | cache.row[cache.row.length - 1] = cache.row[cache.row.length - 1].add(c); 272 | // go to the next for loop 273 | continue; 274 | } 275 | 276 | cache.row.push(c); 277 | 278 | for (var j = 0; j < totalCells; ++j) { 279 | cols.push(parsers[j].format(getElementText(table.config, c[0].cells[j]), table, c[0].cells[j])); 280 | } 281 | 282 | cols.push(cache.normalized.length); // add position for rowCache 283 | cache.normalized.push(cols); 284 | cols = null; 285 | }; 286 | 287 | if (table.config.debug) { 288 | benchmark("Building cache for " + totalRows + " rows:", cacheTime); 289 | } 290 | 291 | return cache; 292 | }; 293 | 294 | function getElementText(config, node) { 295 | 296 | var text = ""; 297 | 298 | if (!node) return ""; 299 | 300 | if (!config.supportsTextContent) config.supportsTextContent = node.textContent || false; 301 | 302 | if (config.textExtraction == "simple") { 303 | if (config.supportsTextContent) { 304 | text = node.textContent; 305 | } else { 306 | if (node.childNodes[0] && node.childNodes[0].hasChildNodes()) { 307 | text = node.childNodes[0].innerHTML; 308 | } else { 309 | text = node.innerHTML; 310 | } 311 | } 312 | } else { 313 | if (typeof(config.textExtraction) == "function") { 314 | text = config.textExtraction(node); 315 | } else { 316 | text = $(node).text(); 317 | } 318 | } 319 | return text; 320 | } 321 | 322 | function appendToTable(table, cache) { 323 | 324 | if (table.config.debug) { 325 | var appendTime = new Date() 326 | } 327 | 328 | var c = cache, 329 | r = c.row, 330 | n = c.normalized, 331 | totalRows = n.length, 332 | checkCell = (n[0].length - 1), 333 | tableBody = $(table.tBodies[0]), 334 | rows = []; 335 | 336 | 337 | for (var i = 0; i < totalRows; i++) { 338 | var pos = n[i][checkCell]; 339 | 340 | rows.push(r[pos]); 341 | 342 | if (!table.config.appender) { 343 | 344 | //var o = ; 345 | var l = r[pos].length; 346 | for (var j = 0; j < l; j++) { 347 | tableBody[0].appendChild(r[pos][j]); 348 | } 349 | 350 | // 351 | } 352 | } 353 | 354 | 355 | 356 | if (table.config.appender) { 357 | 358 | table.config.appender(table, rows); 359 | } 360 | 361 | rows = null; 362 | 363 | if (table.config.debug) { 364 | benchmark("Rebuilt table:", appendTime); 365 | } 366 | 367 | // apply table widgets 368 | applyWidget(table); 369 | 370 | // trigger sortend 371 | setTimeout(function () { 372 | $(table).trigger("sortEnd"); 373 | }, 0); 374 | 375 | }; 376 | 377 | function buildHeaders(table) { 378 | 379 | if (table.config.debug) { 380 | var time = new Date(); 381 | } 382 | 383 | var meta = ($.metadata) ? true : false; 384 | 385 | var header_index = computeTableHeaderCellIndexes(table); 386 | 387 | $tableHeaders = $(table.config.selectorHeaders, table).each(function (index) { 388 | 389 | this.column = header_index[this.parentNode.rowIndex + "-" + this.cellIndex]; 390 | // this.column = index; 391 | this.order = formatSortingOrder(table.config.sortInitialOrder); 392 | 393 | 394 | this.count = this.order; 395 | 396 | if (checkHeaderMetadata(this) || checkHeaderOptions(table, index)) this.sortDisabled = true; 397 | if (checkHeaderOptionsSortingLocked(table, index)) this.order = this.lockedOrder = checkHeaderOptionsSortingLocked(table, index); 398 | 399 | if (!this.sortDisabled) { 400 | var $th = $(this).addClass(table.config.cssHeader); 401 | if (table.config.onRenderHeader) table.config.onRenderHeader.apply($th); 402 | } 403 | 404 | // add cell to headerList 405 | table.config.headerList[index] = this; 406 | }); 407 | 408 | if (table.config.debug) { 409 | benchmark("Built headers:", time); 410 | log($tableHeaders); 411 | } 412 | 413 | return $tableHeaders; 414 | 415 | }; 416 | 417 | // from: 418 | // http://www.javascripttoolbox.com/lib/table/examples.php 419 | // http://www.javascripttoolbox.com/temp/table_cellindex.html 420 | 421 | 422 | function computeTableHeaderCellIndexes(t) { 423 | var matrix = []; 424 | var lookup = {}; 425 | var thead = t.getElementsByTagName('THEAD')[0]; 426 | var trs = thead.getElementsByTagName('TR'); 427 | 428 | for (var i = 0; i < trs.length; i++) { 429 | var cells = trs[i].cells; 430 | for (var j = 0; j < cells.length; j++) { 431 | var c = cells[j]; 432 | 433 | var rowIndex = c.parentNode.rowIndex; 434 | var cellId = rowIndex + "-" + c.cellIndex; 435 | var rowSpan = c.rowSpan || 1; 436 | var colSpan = c.colSpan || 1 437 | var firstAvailCol; 438 | if (typeof(matrix[rowIndex]) == "undefined") { 439 | matrix[rowIndex] = []; 440 | } 441 | // Find first available column in the first row 442 | for (var k = 0; k < matrix[rowIndex].length + 1; k++) { 443 | if (typeof(matrix[rowIndex][k]) == "undefined") { 444 | firstAvailCol = k; 445 | break; 446 | } 447 | } 448 | lookup[cellId] = firstAvailCol; 449 | for (var k = rowIndex; k < rowIndex + rowSpan; k++) { 450 | if (typeof(matrix[k]) == "undefined") { 451 | matrix[k] = []; 452 | } 453 | var matrixrow = matrix[k]; 454 | for (var l = firstAvailCol; l < firstAvailCol + colSpan; l++) { 455 | matrixrow[l] = "x"; 456 | } 457 | } 458 | } 459 | } 460 | return lookup; 461 | } 462 | 463 | function checkCellColSpan(table, rows, row) { 464 | var arr = [], 465 | r = table.tHead.rows, 466 | c = r[row].cells; 467 | 468 | for (var i = 0; i < c.length; i++) { 469 | var cell = c[i]; 470 | 471 | if (cell.colSpan > 1) { 472 | arr = arr.concat(checkCellColSpan(table, headerArr, row++)); 473 | } else { 474 | if (table.tHead.length == 1 || (cell.rowSpan > 1 || !r[row + 1])) { 475 | arr.push(cell); 476 | } 477 | // headerArr[row] = (i+row); 478 | } 479 | } 480 | return arr; 481 | }; 482 | 483 | function checkHeaderMetadata(cell) { 484 | if (($.metadata) && ($(cell).metadata().sorter === false)) { 485 | return true; 486 | }; 487 | return false; 488 | } 489 | 490 | function checkHeaderOptions(table, i) { 491 | if ((table.config.headers[i]) && (table.config.headers[i].sorter === false)) { 492 | return true; 493 | }; 494 | return false; 495 | } 496 | 497 | function checkHeaderOptionsSortingLocked(table, i) { 498 | if ((table.config.headers[i]) && (table.config.headers[i].lockedOrder)) return table.config.headers[i].lockedOrder; 499 | return false; 500 | } 501 | 502 | function applyWidget(table) { 503 | var c = table.config.widgets; 504 | var l = c.length; 505 | for (var i = 0; i < l; i++) { 506 | 507 | getWidgetById(c[i]).format(table); 508 | } 509 | 510 | } 511 | 512 | function getWidgetById(name) { 513 | var l = widgets.length; 514 | for (var i = 0; i < l; i++) { 515 | if (widgets[i].id.toLowerCase() == name.toLowerCase()) { 516 | return widgets[i]; 517 | } 518 | } 519 | }; 520 | 521 | function formatSortingOrder(v) { 522 | if (typeof(v) != "Number") { 523 | return (v.toLowerCase() == "desc") ? 1 : 0; 524 | } else { 525 | return (v == 1) ? 1 : 0; 526 | } 527 | } 528 | 529 | function isValueInArray(v, a) { 530 | var l = a.length; 531 | for (var i = 0; i < l; i++) { 532 | if (a[i][0] == v) { 533 | return true; 534 | } 535 | } 536 | return false; 537 | } 538 | 539 | function setHeadersCss(table, $headers, list, css) { 540 | // remove all header information 541 | $headers.removeClass(css[0]).removeClass(css[1]); 542 | 543 | var h = []; 544 | $headers.each(function (offset) { 545 | if (!this.sortDisabled) { 546 | h[this.column] = $(this); 547 | } 548 | }); 549 | 550 | var l = list.length; 551 | for (var i = 0; i < l; i++) { 552 | h[list[i][0]].addClass(css[list[i][1]]); 553 | } 554 | } 555 | 556 | function fixColumnWidth(table, $headers) { 557 | var c = table.config; 558 | if (c.widthFixed) { 559 | var colgroup = $(''); 560 | $("tr:first td", table.tBodies[0]).each(function () { 561 | colgroup.append($('').css('width', $(this).width())); 562 | }); 563 | $(table).prepend(colgroup); 564 | }; 565 | } 566 | 567 | function updateHeaderSortCount(table, sortList) { 568 | var c = table.config, 569 | l = sortList.length; 570 | for (var i = 0; i < l; i++) { 571 | var s = sortList[i], 572 | o = c.headerList[s[0]]; 573 | o.count = s[1]; 574 | o.count++; 575 | } 576 | } 577 | 578 | /* sorting methods */ 579 | 580 | function multisort(table, sortList, cache) { 581 | 582 | if (table.config.debug) { 583 | var sortTime = new Date(); 584 | } 585 | 586 | var dynamicExp = "var sortWrapper = function(a,b) {", 587 | l = sortList.length; 588 | 589 | // TODO: inline functions. 590 | for (var i = 0; i < l; i++) { 591 | 592 | var c = sortList[i][0]; 593 | var order = sortList[i][1]; 594 | // var s = (getCachedSortType(table.config.parsers,c) == "text") ? 595 | // ((order == 0) ? "sortText" : "sortTextDesc") : ((order == 0) ? 596 | // "sortNumeric" : "sortNumericDesc"); 597 | // var s = (table.config.parsers[c].type == "text") ? ((order == 0) 598 | // ? makeSortText(c) : makeSortTextDesc(c)) : ((order == 0) ? 599 | // makeSortNumeric(c) : makeSortNumericDesc(c)); 600 | var s = (table.config.parsers[c].type == "text") ? ((order == 0) ? makeSortFunction("text", "asc", c) : makeSortFunction("text", "desc", c)) : ((order == 0) ? makeSortFunction("numeric", "asc", c) : makeSortFunction("numeric", "desc", c)); 601 | var e = "e" + i; 602 | 603 | dynamicExp += "var " + e + " = " + s; // + "(a[" + c + "],b[" + c 604 | // + "]); "; 605 | dynamicExp += "if(" + e + ") { return " + e + "; } "; 606 | dynamicExp += "else { "; 607 | 608 | } 609 | 610 | // if value is the same keep orignal order 611 | var orgOrderCol = cache.normalized[0].length - 1; 612 | dynamicExp += "return a[" + orgOrderCol + "]-b[" + orgOrderCol + "];"; 613 | 614 | for (var i = 0; i < l; i++) { 615 | dynamicExp += "}; "; 616 | } 617 | 618 | dynamicExp += "return 0; "; 619 | dynamicExp += "}; "; 620 | 621 | if (table.config.debug) { 622 | benchmark("Evaling expression:" + dynamicExp, new Date()); 623 | } 624 | 625 | eval(dynamicExp); 626 | 627 | cache.normalized.sort(sortWrapper); 628 | 629 | if (table.config.debug) { 630 | benchmark("Sorting on " + sortList.toString() + " and dir " + order + " time:", sortTime); 631 | } 632 | 633 | return cache; 634 | }; 635 | 636 | function makeSortFunction(type, direction, index) { 637 | var a = "a[" + index + "]", 638 | b = "b[" + index + "]"; 639 | if (type == 'text' && direction == 'asc') { 640 | return "(" + a + " == " + b + " ? 0 : (" + a + " === null ? Number.POSITIVE_INFINITY : (" + b + " === null ? Number.NEGATIVE_INFINITY : (" + a + " < " + b + ") ? -1 : 1 )));"; 641 | } else if (type == 'text' && direction == 'desc') { 642 | return "(" + a + " == " + b + " ? 0 : (" + a + " === null ? Number.POSITIVE_INFINITY : (" + b + " === null ? Number.NEGATIVE_INFINITY : (" + b + " < " + a + ") ? -1 : 1 )));"; 643 | } else if (type == 'numeric' && direction == 'asc') { 644 | return "(" + a + " === null && " + b + " === null) ? 0 :(" + a + " === null ? Number.POSITIVE_INFINITY : (" + b + " === null ? Number.NEGATIVE_INFINITY : " + a + " - " + b + "));"; 645 | } else if (type == 'numeric' && direction == 'desc') { 646 | return "(" + a + " === null && " + b + " === null) ? 0 :(" + a + " === null ? Number.POSITIVE_INFINITY : (" + b + " === null ? Number.NEGATIVE_INFINITY : " + b + " - " + a + "));"; 647 | } 648 | }; 649 | 650 | function makeSortText(i) { 651 | return "((a[" + i + "] < b[" + i + "]) ? -1 : ((a[" + i + "] > b[" + i + "]) ? 1 : 0));"; 652 | }; 653 | 654 | function makeSortTextDesc(i) { 655 | return "((b[" + i + "] < a[" + i + "]) ? -1 : ((b[" + i + "] > a[" + i + "]) ? 1 : 0));"; 656 | }; 657 | 658 | function makeSortNumeric(i) { 659 | return "a[" + i + "]-b[" + i + "];"; 660 | }; 661 | 662 | function makeSortNumericDesc(i) { 663 | return "b[" + i + "]-a[" + i + "];"; 664 | }; 665 | 666 | function sortText(a, b) { 667 | if (table.config.sortLocaleCompare) return a.localeCompare(b); 668 | return ((a < b) ? -1 : ((a > b) ? 1 : 0)); 669 | }; 670 | 671 | function sortTextDesc(a, b) { 672 | if (table.config.sortLocaleCompare) return b.localeCompare(a); 673 | return ((b < a) ? -1 : ((b > a) ? 1 : 0)); 674 | }; 675 | 676 | function sortNumeric(a, b) { 677 | return a - b; 678 | }; 679 | 680 | function sortNumericDesc(a, b) { 681 | return b - a; 682 | }; 683 | 684 | function getCachedSortType(parsers, i) { 685 | return parsers[i].type; 686 | }; /* public methods */ 687 | this.construct = function (settings) { 688 | return this.each(function () { 689 | // if no thead or tbody quit. 690 | if (!this.tHead || !this.tBodies) return; 691 | // declare 692 | var $this, $document, $headers, cache, config, shiftDown = 0, 693 | sortOrder; 694 | // new blank config object 695 | this.config = {}; 696 | // merge and extend. 697 | config = $.extend(this.config, $.tablesorter.defaults, settings); 698 | // store common expression for speed 699 | $this = $(this); 700 | // save the settings where they read 701 | $.data(this, "tablesorter", config); 702 | // build headers 703 | $headers = buildHeaders(this); 704 | // try to auto detect column type, and store in tables config 705 | this.config.parsers = buildParserCache(this, $headers); 706 | // build the cache for the tbody cells 707 | cache = buildCache(this); 708 | // get the css class names, could be done else where. 709 | var sortCSS = [config.cssDesc, config.cssAsc]; 710 | // fixate columns if the users supplies the fixedWidth option 711 | fixColumnWidth(this); 712 | // apply event handling to headers 713 | // this is to big, perhaps break it out? 714 | $headers.click( 715 | 716 | function (e) { 717 | var totalRows = ($this[0].tBodies[0] && $this[0].tBodies[0].rows.length) || 0; 718 | if (!this.sortDisabled && totalRows > 0) { 719 | // Only call sortStart if sorting is 720 | // enabled. 721 | $this.trigger("sortStart"); 722 | // store exp, for speed 723 | var $cell = $(this); 724 | // get current column index 725 | var i = this.column; 726 | // get current column sort order 727 | this.order = this.count++ % 2; 728 | // always sort on the locked order. 729 | if(this.lockedOrder) this.order = this.lockedOrder; 730 | 731 | // user only whants to sort on one 732 | // column 733 | if (!e[config.sortMultiSortKey]) { 734 | // flush the sort list 735 | config.sortList = []; 736 | if (config.sortForce != null) { 737 | var a = config.sortForce; 738 | for (var j = 0; j < a.length; j++) { 739 | if (a[j][0] != i) { 740 | config.sortList.push(a[j]); 741 | } 742 | } 743 | } 744 | // add column to sort list 745 | config.sortList.push([i, this.order]); 746 | // multi column sorting 747 | } else { 748 | // the user has clicked on an all 749 | // ready sortet column. 750 | if (isValueInArray(i, config.sortList)) { 751 | // revers the sorting direction 752 | // for all tables. 753 | for (var j = 0; j < config.sortList.length; j++) { 754 | var s = config.sortList[j], 755 | o = config.headerList[s[0]]; 756 | if (s[0] == i) { 757 | o.count = s[1]; 758 | o.count++; 759 | s[1] = o.count % 2; 760 | } 761 | } 762 | } else { 763 | // add column to sort list array 764 | config.sortList.push([i, this.order]); 765 | } 766 | }; 767 | setTimeout(function () { 768 | // set css for headers 769 | setHeadersCss($this[0], $headers, config.sortList, sortCSS); 770 | appendToTable( 771 | $this[0], multisort( 772 | $this[0], config.sortList, cache) 773 | ); 774 | }, 1); 775 | // stop normal event by returning false 776 | return false; 777 | } 778 | // cancel selection 779 | }).mousedown(function () { 780 | if (config.cancelSelection) { 781 | this.onselectstart = function () { 782 | return false 783 | }; 784 | return false; 785 | } 786 | }); 787 | // apply easy methods that trigger binded events 788 | $this.bind("update", function () { 789 | var me = this; 790 | setTimeout(function () { 791 | // rebuild parsers. 792 | me.config.parsers = buildParserCache( 793 | me, $headers); 794 | // rebuild the cache map 795 | cache = buildCache(me); 796 | }, 1); 797 | }).bind("updateCell", function (e, cell) { 798 | var config = this.config; 799 | // get position from the dom. 800 | var pos = [(cell.parentNode.rowIndex - 1), cell.cellIndex]; 801 | // update cache 802 | cache.normalized[pos[0]][pos[1]] = config.parsers[pos[1]].format( 803 | getElementText(config, cell), cell); 804 | }).bind("sorton", function (e, list) { 805 | $(this).trigger("sortStart"); 806 | config.sortList = list; 807 | // update and store the sortlist 808 | var sortList = config.sortList; 809 | // update header count index 810 | updateHeaderSortCount(this, sortList); 811 | // set css for headers 812 | setHeadersCss(this, $headers, sortList, sortCSS); 813 | // sort the table and append it to the dom 814 | appendToTable(this, multisort(this, sortList, cache)); 815 | }).bind("appendCache", function () { 816 | appendToTable(this, cache); 817 | }).bind("applyWidgetId", function (e, id) { 818 | getWidgetById(id).format(this); 819 | }).bind("applyWidgets", function () { 820 | // apply widgets 821 | applyWidget(this); 822 | }); 823 | if ($.metadata && ($(this).metadata() && $(this).metadata().sortlist)) { 824 | config.sortList = $(this).metadata().sortlist; 825 | } 826 | // if user has supplied a sort list to constructor. 827 | if (config.sortList.length > 0) { 828 | $this.trigger("sorton", [config.sortList]); 829 | } 830 | // apply widgets 831 | applyWidget(this); 832 | }); 833 | }; 834 | this.addParser = function (parser) { 835 | var l = parsers.length, 836 | a = true; 837 | for (var i = 0; i < l; i++) { 838 | if (parsers[i].id.toLowerCase() == parser.id.toLowerCase()) { 839 | a = false; 840 | } 841 | } 842 | if (a) { 843 | parsers.push(parser); 844 | }; 845 | }; 846 | this.addWidget = function (widget) { 847 | widgets.push(widget); 848 | }; 849 | this.formatFloat = function (s) { 850 | var i = parseFloat(s); 851 | return (isNaN(i)) ? 0 : i; 852 | }; 853 | this.formatInt = function (s) { 854 | var i = parseInt(s); 855 | return (isNaN(i)) ? 0 : i; 856 | }; 857 | this.isDigit = function (s, config) { 858 | // replace all an wanted chars and match. 859 | return /^[-+]?\d*$/.test($.trim(s.replace(/[,.']/g, ''))); 860 | }; 861 | this.clearTableBody = function (table) { 862 | if ($.browser.msie) { 863 | function empty() { 864 | while (this.firstChild) 865 | this.removeChild(this.firstChild); 866 | } 867 | empty.apply(table.tBodies[0]); 868 | } else { 869 | table.tBodies[0].innerHTML = ""; 870 | } 871 | }; 872 | } 873 | }); 874 | 875 | // extend plugin scope 876 | $.fn.extend({ 877 | tablesorter: $.tablesorter.construct 878 | }); 879 | 880 | // make shortcut 881 | var ts = $.tablesorter; 882 | 883 | // add default parsers 884 | ts.addParser({ 885 | id: "text", 886 | is: function (s) { 887 | return true; 888 | }, format: function (s) { 889 | return $.trim(s.toLocaleLowerCase()); 890 | }, type: "text" 891 | }); 892 | 893 | ts.addParser({ 894 | id: "digit", 895 | is: function (s, table) { 896 | var c = table.config; 897 | return $.tablesorter.isDigit(s, c); 898 | }, format: function (s) { 899 | return $.tablesorter.formatFloat(s); 900 | }, type: "numeric" 901 | }); 902 | 903 | ts.addParser({ 904 | id: "currency", 905 | is: function (s) { 906 | return /^[£$€?.]/.test(s); 907 | }, format: function (s) { 908 | return $.tablesorter.formatFloat(s.replace(new RegExp(/[£$€]/g), "")); 909 | }, type: "numeric" 910 | }); 911 | 912 | ts.addParser({ 913 | id: "ipAddress", 914 | is: function (s) { 915 | return /^\d{2,3}[\.]\d{2,3}[\.]\d{2,3}[\.]\d{2,3}$/.test(s); 916 | }, format: function (s) { 917 | var a = s.split("."), 918 | r = "", 919 | l = a.length; 920 | for (var i = 0; i < l; i++) { 921 | var item = a[i]; 922 | if (item.length == 2) { 923 | r += "0" + item; 924 | } else { 925 | r += item; 926 | } 927 | } 928 | return $.tablesorter.formatFloat(r); 929 | }, type: "numeric" 930 | }); 931 | 932 | ts.addParser({ 933 | id: "url", 934 | is: function (s) { 935 | return /^(https?|ftp|file):\/\/$/.test(s); 936 | }, format: function (s) { 937 | return jQuery.trim(s.replace(new RegExp(/(https?|ftp|file):\/\//), '')); 938 | }, type: "text" 939 | }); 940 | 941 | ts.addParser({ 942 | id: "isoDate", 943 | is: function (s) { 944 | return /^\d{4}[\/-]\d{1,2}[\/-]\d{1,2}$/.test(s); 945 | }, format: function (s) { 946 | return $.tablesorter.formatFloat((s != "") ? new Date(s.replace( 947 | new RegExp(/-/g), "/")).getTime() : "0"); 948 | }, type: "numeric" 949 | }); 950 | 951 | ts.addParser({ 952 | id: "percent", 953 | is: function (s) { 954 | return /\%$/.test($.trim(s)); 955 | }, format: function (s) { 956 | return $.tablesorter.formatFloat(s.replace(new RegExp(/%/g), "")); 957 | }, type: "numeric" 958 | }); 959 | 960 | ts.addParser({ 961 | id: "usLongDate", 962 | is: function (s) { 963 | return s.match(new RegExp(/^[A-Za-z]{3,10}\.? [0-9]{1,2}, ([0-9]{4}|'?[0-9]{2}) (([0-2]?[0-9]:[0-5][0-9])|([0-1]?[0-9]:[0-5][0-9]\s(AM|PM)))$/)); 964 | }, format: function (s) { 965 | return $.tablesorter.formatFloat(new Date(s).getTime()); 966 | }, type: "numeric" 967 | }); 968 | 969 | ts.addParser({ 970 | id: "shortDate", 971 | is: function (s) { 972 | return /\d{1,2}[\/\-]\d{1,2}[\/\-]\d{2,4}/.test(s); 973 | }, format: function (s, table) { 974 | var c = table.config; 975 | s = s.replace(/\-/g, "/"); 976 | if (c.dateFormat == "us") { 977 | // reformat the string in ISO format 978 | s = s.replace(/(\d{1,2})[\/\-](\d{1,2})[\/\-](\d{4})/, "$3/$1/$2"); 979 | } else if (c.dateFormat == "uk") { 980 | // reformat the string in ISO format 981 | s = s.replace(/(\d{1,2})[\/\-](\d{1,2})[\/\-](\d{4})/, "$3/$2/$1"); 982 | } else if (c.dateFormat == "dd/mm/yy" || c.dateFormat == "dd-mm-yy") { 983 | s = s.replace(/(\d{1,2})[\/\-](\d{1,2})[\/\-](\d{2})/, "$1/$2/$3"); 984 | } 985 | return $.tablesorter.formatFloat(new Date(s).getTime()); 986 | }, type: "numeric" 987 | }); 988 | ts.addParser({ 989 | id: "time", 990 | is: function (s) { 991 | return /^(([0-2]?[0-9]:[0-5][0-9])|([0-1]?[0-9]:[0-5][0-9]\s(am|pm)))$/.test(s); 992 | }, format: function (s) { 993 | return $.tablesorter.formatFloat(new Date("2000/01/01 " + s).getTime()); 994 | }, type: "numeric" 995 | }); 996 | ts.addParser({ 997 | id: "metadata", 998 | is: function (s) { 999 | return false; 1000 | }, format: function (s, table, cell) { 1001 | var c = table.config, 1002 | p = (!c.parserMetadataName) ? 'sortValue' : c.parserMetadataName; 1003 | return $(cell).metadata()[p]; 1004 | }, type: "numeric" 1005 | }); 1006 | // add default widgets 1007 | ts.addWidget({ 1008 | id: "zebra", 1009 | format: function (table) { 1010 | if (table.config.debug) { 1011 | var time = new Date(); 1012 | } 1013 | var $tr, row = -1, 1014 | odd; 1015 | // loop through the visible rows 1016 | $("tr:visible", table.tBodies[0]).each(function (i) { 1017 | $tr = $(this); 1018 | // style children rows the same way the parent 1019 | // row was styled 1020 | if (!$tr.hasClass(table.config.cssChildRow)) row++; 1021 | odd = (row % 2 == 0); 1022 | $tr.removeClass( 1023 | table.config.widgetZebra.css[odd ? 0 : 1]).addClass( 1024 | table.config.widgetZebra.css[odd ? 1 : 0]) 1025 | }); 1026 | if (table.config.debug) { 1027 | $.tablesorter.benchmark("Applying Zebra widget", time); 1028 | } 1029 | } 1030 | }); 1031 | })(jQuery); -------------------------------------------------------------------------------- /data/exaddos/html/inc/jquery.tmpl.min.js: -------------------------------------------------------------------------------- 1 | /* 2 | * jQuery Templates Plugin 1.0.0pre 3 | * http://github.com/jquery/jquery-tmpl 4 | * Requires jQuery 1.4.2 5 | * 6 | * Copyright 2011, Software Freedom Conservancy, Inc. 7 | * Dual licensed under the MIT or GPL Version 2 licenses. 8 | * http://jquery.org/license 9 | */ 10 | (function(a){var r=a.fn.domManip,d="_tmplitem",q=/^[^<]*(<[\w\W]+>)[^>]*$|\{\{\! /,b={},f={},e,p={key:0,data:{}},i=0,c=0,l=[];function g(g,d,h,e){var c={data:e||(e===0||e===false)?e:d?d.data:{},_wrap:d?d._wrap:null,tmpl:null,parent:d||null,nodes:[],calls:u,nest:w,wrap:x,html:v,update:t};g&&a.extend(c,g,{nodes:[],parent:d});if(h){c.tmpl=h;c._ctnt=c._ctnt||c.tmpl(a,c);c.key=++i;(l.length?f:b)[i]=c}return c}a.each({appendTo:"append",prependTo:"prepend",insertBefore:"before",insertAfter:"after",replaceAll:"replaceWith"},function(f,d){a.fn[f]=function(n){var g=[],i=a(n),k,h,m,l,j=this.length===1&&this[0].parentNode;e=b||{};if(j&&j.nodeType===11&&j.childNodes.length===1&&i.length===1){i[d](this[0]);g=this}else{for(h=0,m=i.length;h0?this.clone(true):this).get();a(i[h])[d](k);g=g.concat(k)}c=0;g=this.pushStack(g,f,i.selector)}l=e;e=null;a.tmpl.complete(l);return g}});a.fn.extend({tmpl:function(d,c,b){return a.tmpl(this[0],d,c,b)},tmplItem:function(){return a.tmplItem(this[0])},template:function(b){return a.template(b,this[0])},domManip:function(d,m,k){if(d[0]&&a.isArray(d[0])){var g=a.makeArray(arguments),h=d[0],j=h.length,i=0,f;while(i").join(">").split('"').join(""").split("'").join("'")}});a.extend(a.tmpl,{tag:{tmpl:{_default:{$2:"null"},open:"if($notnull_1){__=__.concat($item.nest($1,$2));}"},wrap:{_default:{$2:"null"},open:"$item.calls(__,$1,$2);__=[];",close:"call=$item.calls();__=call._.concat($item.wrap(call,__));"},each:{_default:{$2:"$index, $value"},open:"if($notnull_1){$.each($1a,function($2){with(this){",close:"}});}"},"if":{open:"if(($notnull_1) && $1a){",close:"}"},"else":{_default:{$1:"true"},open:"}else if(($notnull_1) && $1a){"},html:{open:"if($notnull_1){__.push($1a);}"},"=":{_default:{$1:"$data"},open:"if($notnull_1){__.push($.encode($1a));}"},"!":{open:""}},complete:function(){b={}},afterManip:function(f,b,d){var e=b.nodeType===11?a.makeArray(b.childNodes):b.nodeType===1?[b]:[];d.call(f,b);m(e);c++}});function j(e,g,f){var b,c=f?a.map(f,function(a){return typeof a==="string"?e.key?a.replace(/(<\w+)(?=[\s>])(?![^>]*_tmplitem)([^>]*)/g,"$1 "+d+'="'+e.key+'" $2'):a:j(a,e,a._ctnt)}):e;if(g)return c;c=c.join("");c.replace(/^\s*([^<\s][^<]*)?(<[\w\W]+>)([^>]*[^>\s])?\s*$/,function(f,c,e,d){b=a(e).get();m(b);if(c)b=k(c).concat(b);if(d)b=b.concat(k(d))});return b?b:k(c)}function k(c){var b=document.createElement("div");b.innerHTML=c;return a.makeArray(b.childNodes)}function o(b){return new Function("jQuery","$item","var $=jQuery,call,__=[],$data=$item.data;with($data){__.push('"+a.trim(b).replace(/([\\'])/g,"\\$1").replace(/[\r\t\n]/g," ").replace(/\$\{([^\}]*)\}/g,"{{= $1}}").replace(/\{\{(\/?)(\w+|.)(?:\(((?:[^\}]|\}(?!\}))*?)?\))?(?:\s+(.*?)?)?(\(((?:[^\}]|\}(?!\}))*?)\))?\s*\}\}/g,function(m,l,k,g,b,c,d){var j=a.tmpl.tag[k],i,e,f;if(!j)throw"Unknown template tag: "+k;i=j._default||[];if(c&&!/\w$/.test(b)){b+=c;c=""}if(b){b=h(b);d=d?","+h(d)+")":c?")":"";e=c?b.indexOf(".")>-1?b+h(c):"("+b+").call($item"+d:b;f=c?e:"(typeof("+b+")==='function'?("+b+").call($item):("+b+"))"}else f=e=i.$1||"null";g=h(g);return"');"+j[l?"close":"open"].split("$notnull_1").join(b?"typeof("+b+")!=='undefined' && ("+b+")!=null":"true").split("$1a").join(f).split("$1").join(e).split("$2").join(g||i.$2||"")+"__.push('"})+"');}return __;")}function n(c,b){c._wrap=j(c,true,a.isArray(b)?b:[q.test(b)?b:a(b).html()]).join("")}function h(a){return a?a.replace(/\\'/g,"'").replace(/\\\\/g,"\\"):null}function s(b){var a=document.createElement("div");a.appendChild(b.cloneNode(true));return a.innerHTML}function m(o){var n="_"+c,k,j,l={},e,p,h;for(e=0,p=o.length;e=0;h--)m(j[h]);m(k)}function m(j){var p,h=j,k,e,m;if(m=j.getAttribute(d)){while(h.parentNode&&(h=h.parentNode).nodeType===1&&!(p=h.getAttribute(d)));if(p!==m){h=h.parentNode?h.nodeType===11?0:h.getAttribute(d)||0:0;if(!(e=b[m])){e=f[m];e=g(e,b[h]||f[h]);e.key=++i;b[i]=e}c&&o(m)}j.removeAttribute(d)}else if(c&&(e=a.data(j,"tmplItem"))){o(e.key);b[e.key]=e;h=a.data(j.parentNode,"tmplItem");h=h?h.key:0}if(e){k=e;while(k&&k.key!=h){k.nodes.push(j);k=k.parent}delete e._ctnt;delete e._wrap;a.data(j,"tmplItem",e)}function o(a){a=a+n;e=l[a]=l[a]||g(e,b[e.parent.key+n]||e.parent)}}}function u(a,d,c,b){if(!a)return l.pop();l.push({_:a,tmpl:d,item:this,data:c,options:b})}function w(d,c,b){return a.tmpl(a.template(d),c,b,this)}function x(b,d){var c=b.options||{};c.wrapped=d;return a.tmpl(a.template(b.tmpl),b.data,c,b.item)}function v(d,c){var b=this._wrap;return a.map(a(a.isArray(b)?b.join(""):b).filter(d||"*"),function(a){return c?a.innerText||a.textContent:a.outerHTML||s(a)})}function t(){var b=this.nodes;a.tmpl(null,null,null,this).insertBefore(b[0]);a(b).remove()}})(jQuery); -------------------------------------------------------------------------------- /data/exaddos/html/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /data/exaddos/html/overview.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 142 | 143 | 144 | 145 | 146 | 147 | 254 | 255 | 260 | 261 | 273 | 274 | 283 | 284 | 285 | 286 | 287 | 288 | 289 | 290 | 296 | 297 |
298 | 299 | 300 | 301 | 302 | 303 | 304 | 305 | 306 | 307 | 308 | 309 | 310 | 311 | 312 | 313 | 314 |
NodeLocationBandwidthUnicastNonUnicastDropErrorQuery Time
315 | 316 | 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | 327 | 328 |
CounterUDPTCPICMPTotal
329 | 330 | 331 | 332 | -------------------------------------------------------------------------------- /data/exaddos/html/talker.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 209 | 210 | 215 | 216 | 235 | 236 | 247 | 248 | 249 | 250 | 251 | 252 | 253 | 254 | 255 |
256 | 257 |
258 | 259 | 260 | 261 | -------------------------------------------------------------------------------- /data/exaddos/html/talkers.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 122 | 123 | 124 | 125 | 126 | 223 | 224 | 229 | 230 | 236 | 237 | 238 | 239 | 240 | 241 | 242 | 243 | 249 | 250 |
251 | 252 |
253 | 254 | 255 | 256 | 257 | 258 | 259 | 260 | 261 | 262 | 263 |
Src IPPackets
264 | 265 | 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 |
Src IPBandwidth
275 | 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | 284 | 285 |
Src IPFlows
286 | 287 |
288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 | 297 | 298 |
Src PortPackets
299 | 300 | 301 | 302 | 303 | 304 | 305 | 306 | 307 | 308 | 309 |
Src PortBandwidth
310 | 311 | 312 | 313 | 314 | 315 | 316 | 317 | 318 | 319 | 320 |
Src PortFlows
321 | 322 |
323 | 324 | 325 | 326 | 327 | 328 | 329 | 330 | 331 | 332 | 333 |
Dst IPPackets
334 | 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | 343 | 344 |
Dst IPBandwidth
345 | 346 | 347 | 348 | 349 | 350 | 351 | 352 | 353 | 354 | 355 |
Dst IPFlows
356 | 357 |
358 | 359 | 360 | 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 |
Dst PortPackets
369 | 370 | 371 | 372 | 373 | 374 | 375 | 376 | 377 | 378 | 379 |
Dst PortBandwidth
380 | 381 | 382 | 383 | 384 | 385 | 386 | 387 | 388 | 389 | 390 |
Dst PortFlows
391 | 392 | 393 | 394 | -------------------------------------------------------------------------------- /etc/exaddos/exaddos.conf.snmpv2: -------------------------------------------------------------------------------- 1 | [exaddos.daemon] 2 | user = 'nobody' 3 | 4 | [exaddos.http] 5 | host = '127.0.0.1' 6 | port = 8080 7 | 8 | [exaddos.ipfix] 9 | host = '127.0.0.1' 10 | port = 29300 11 | 12 | 13 | [exaddos.location] 14 | #database = 'data/exaddos/db/exaddos.sqlite3' 15 | html = 'data/exaddos/html' 16 | 17 | [exaddos.profile] 18 | destination = 'stdout' 19 | enable = false 20 | 21 | # threshold: expressed in pps 22 | 23 | [exaddos.PEER1] 24 | router = '127.0.0.2' 25 | snmp_version = 2 26 | snmp_password = 'secret' 27 | snmp_frequency = 10 28 | snmp_index_port = 120 29 | snmp_index_vlan = 100 30 | threshold_bandwidth = 104857600 31 | threshold_unicast = 120000 32 | threshold_notunicast = 1500 33 | 34 | 35 | [exaddos.PEER2] 36 | router = '127.0.0.2' 37 | snmp_version = 2 38 | snmp_password = 'secret' 39 | snmp_frequency = 10 40 | snmp_index_port = 120 41 | snmp_index_vlan = 110 42 | threshold_bandwidth = 104857600 43 | threshold_unicast = 120000 44 | threshold_notunicast = 1500 45 | 46 | 47 | [exaddos.TRANSIT1] 48 | router = '127.0.0.3' 49 | location = 'datacenter' 50 | snmp_version = 2 51 | snmp_password = 'secret' 52 | snmp_frequency = 10 53 | snmp_index_port = 220 54 | snmp_index_vlan = 140 55 | threshold_bandwidth = 104857600 56 | threshold_unicast = 120000 57 | threshold_notunicast = 1500 58 | -------------------------------------------------------------------------------- /etc/exaddos/exaddos.conf.snmpv3: -------------------------------------------------------------------------------- 1 | [exaddos.daemon] 2 | user = 'nobody' 3 | 4 | [exaddos.http] 5 | host = '127.0.0.1' 6 | #host = '192.0.2.201' 7 | port = 8080 8 | 9 | [exaddos.ipfix] 10 | host = '127.0.0.1' 11 | port = 29300 12 | 13 | [exaddos.location] 14 | #database = 'data/exaddos/db/exaddos.sqlite3' 15 | html = 'data/exaddos/html' 16 | 17 | [exaddos.profile] 18 | destination = 'stdout' 19 | enable = false 20 | 21 | [exaddos.router] 22 | router = '192.0.2.123' 23 | snmp_version = 3 24 | snmp_user = 'exaddos' 25 | snmp_auth_method = 'SHA' 26 | snmp_auth_key = 'pass7faYDFagREF6DasdG' 27 | snmp_privacy_method = 'AES-128' 28 | snmp_privacy_key = 'privTasvRT3gfg4GFsGHV' 29 | snmp_frequency = 10 30 | snmp_index_port = 520 31 | snmp_index_vlan = 573 32 | threshold_bandwidth = 524288000 33 | threshold_unicast = 120000 34 | threshold_notunicast = 1500 35 | -------------------------------------------------------------------------------- /lib/exaddos/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Exa-Networks/exaddos/15c5e0a5575bab4c8fbac31eeb6790eb9b710a39/lib/exaddos/__init__.py -------------------------------------------------------------------------------- /lib/exaddos/application.py: -------------------------------------------------------------------------------- 1 | """ 2 | application.py 3 | 4 | Created by Thomas Mangin on 2014-02-06. 5 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 6 | """ 7 | 8 | import os 9 | import sys 10 | import pwd 11 | 12 | import socket 13 | import errno 14 | 15 | import atexit 16 | 17 | from log import log,err,silence 18 | from exaddos import reactor 19 | 20 | def __exit(memory,code): 21 | if memory: 22 | from exaddos.leak import objgraph 23 | print "memory utilisation" 24 | print 25 | print objgraph.show_most_common_types(limit=20) 26 | print 27 | print 28 | print "generating memory utilisation graph" 29 | print 30 | obj = objgraph.by_type('run') 31 | objgraph.show_backrefs([obj], max_depth=10) 32 | sys.exit(code) 33 | 34 | 35 | def __drop_privileges (user): 36 | """returns true if we are left with insecure privileges""" 37 | try: 38 | user = pwd.getpwnam(user) 39 | nuid = int(user.pw_uid) 40 | ngid = int(user.pw_gid) 41 | except KeyError: 42 | return False 43 | 44 | uid = os.getuid() 45 | gid = os.getgid() 46 | 47 | # not sure you can change your gid if you do not have a pid of zero 48 | try: 49 | # we must change the GID first otherwise it may fail after change UID 50 | if not gid: 51 | os.setgid(ngid) 52 | if not uid: 53 | os.setuid(nuid) 54 | 55 | cuid = os.getuid() 56 | ceid = os.geteuid() 57 | cgid = os.getgid() 58 | 59 | if cuid < 0: 60 | cuid = (1<<32) + cuid 61 | 62 | if cgid < 0: 63 | cgid = (1<<32) + cgid 64 | 65 | if ceid < 0: 66 | ceid = (1<<32) + ceid 67 | 68 | if nuid != cuid or nuid != ceid or ngid != cgid: 69 | return False 70 | 71 | except OSError: 72 | return False 73 | 74 | return True 75 | 76 | def drop_privileges (configuration): 77 | # os.name can be ['posix', 'nt', 'os2', 'ce', 'java', 'riscos'] 78 | if os.name not in ['posix',]: 79 | return True 80 | 81 | if os.getuid() != 0: 82 | err('not running as root, not changing UID') 83 | return True 84 | 85 | users = [configuration.daemon.user,'nobody'] 86 | 87 | for user in users: 88 | if __drop_privileges(user): 89 | return True 90 | return False 91 | 92 | def daemonise (daemonize): 93 | if not daemonize: 94 | return 95 | 96 | def fork_exit (): 97 | try: 98 | pid = os.fork() 99 | if pid > 0: 100 | os._exit(0) 101 | except OSError, e: 102 | err('Can not fork, errno %d : %s' % (e.errno,e.strerror)) 103 | 104 | def mute (): 105 | # closing more would close the log file too if open 106 | maxfd = 3 107 | 108 | for fd in range(0, maxfd): 109 | try: 110 | os.close(fd) 111 | except OSError: 112 | pass 113 | os.open("/dev/null", os.O_RDWR) 114 | os.dup2(0, 1) 115 | os.dup2(0, 2) 116 | 117 | def is_socket (fd): 118 | try: 119 | s = socket.fromfd(fd, socket.AF_INET, socket.SOCK_RAW) 120 | except ValueError,e: 121 | # The file descriptor is closed 122 | return False 123 | try: 124 | s.getsockopt(socket.SOL_SOCKET, socket.SO_TYPE) 125 | except socket.error, e: 126 | # It is look like one but it is not a socket ... 127 | if e.args[0] == errno.ENOTSOCK: 128 | return False 129 | return True 130 | 131 | # do not detach if we are already supervised or run by init like process 132 | if is_socket(sys.__stdin__.fileno()) or os.getppid() == 1: 133 | return 134 | 135 | fork_exit() 136 | os.setsid() 137 | fork_exit() 138 | mute() 139 | silence() 140 | 141 | def savepid (location): 142 | if not location: 143 | return 144 | 145 | ownid = os.getpid() 146 | 147 | flags = os.O_CREAT | os.O_EXCL | os.O_WRONLY 148 | mode = ((os.R_OK | os.W_OK) << 6) | (os.R_OK << 3) | os.R_OK 149 | 150 | try: 151 | fd = os.open(location,flags,mode) 152 | except OSError: 153 | err("PIDfile already exists, not updated %s" % location) 154 | return False 155 | 156 | try: 157 | f = os.fdopen(fd,'w') 158 | line = "%d\n" % ownid 159 | f.write(line) 160 | f.close() 161 | except IOError: 162 | err("Can not create PIDfile %s" % location) 163 | return False 164 | log("Created PIDfile %s with value %d" % (location,ownid)) 165 | atexit.register(removepid,location) 166 | return True 167 | 168 | def removepid (location): 169 | if not location: 170 | return 171 | 172 | try: 173 | os.remove(location) 174 | except OSError, e: 175 | if e.errno == errno.ENOENT: 176 | pass 177 | else: 178 | err("Can not remove PIDfile %s" % location) 179 | return 180 | log("Removed PIDfile %s" % location) 181 | 182 | 183 | def help (): 184 | sys.stdout.write('usage:\n exaddos [options]\n') 185 | sys.stdout.write('\n') 186 | sys.stdout.write(' -h, --help : this help\n') 187 | sys.stdout.write(' -c, --conf-file : configuration file to use (ini format)\n') 188 | sys.stdout.write(' -i, --ini : display the configuration using the ini format\n') 189 | sys.stdout.write(' -e, --env : display the configuration using the env format\n') 190 | sys.stdout.write(' -di, --diff-ini : display non-default configurations values using the ini format\n') 191 | sys.stdout.write(' -de, --diff-env : display non-default configurations values using the env format\n') 192 | sys.stdout.write(' -d, --debug : shortcut to turn on all subsystems debugging to LOG_DEBUG\n') 193 | sys.stdout.write(' -p, --pdb : start the python debugger on serious logging and on SIGTERM\n') 194 | sys.stdout.write(' -m, --memory : display memory usage information on exit\n') 195 | 196 | sys.stdout.write('\n') 197 | sys.stdout.write('iEnum will automatically look for its configuration file (in windows ini format)\n') 198 | sys.stdout.write(' - in the etc/exaddos folder located within the extracted tar.gz \n') 199 | sys.stdout.write(' - in /etc/exaddos/exaddos.conf\n') 200 | sys.stdout.write('\n') 201 | sys.stdout.write('Every configuration value has a sensible built-in default\n') 202 | sys.stdout.write('\n') 203 | sys.stdout.write('Individual configuration options can be set using environment variables, such as :\n') 204 | sys.stdout.write(' > env exaddos.http.port=39200 ./sbin/exaddos\n') 205 | sys.stdout.write('or > env exaddos_http_port=39200 ./sbin/exaddos\n') 206 | sys.stdout.write('or > export exaddos_http_port=39200; ./sbin/exaddos\n') 207 | sys.stdout.write('\n') 208 | sys.stdout.write('Multiple environment values can be set\n') 209 | sys.stdout.write('and the order of preference is :\n') 210 | sys.stdout.write(' - 1 : command line env value using dot separated notation\n') 211 | sys.stdout.write(' - 2 : exported value from the shell using dot separated notation\n') 212 | sys.stdout.write(' - 3 : command line env value using underscore separated notation\n') 213 | sys.stdout.write(' - 4 : exported value from the shell using underscore separated notation\n') 214 | sys.stdout.write(' - 5 : the value in the ini configuration file\n') 215 | sys.stdout.write('\n') 216 | sys.stdout.write('Valid configuration options are :\n') 217 | sys.stdout.write('\n') 218 | for line in default(): 219 | sys.stdout.write(' - %s\n' % line) 220 | sys.stdout.write('\n') 221 | 222 | def version_warning (): 223 | sys.stderr.write('This version of python is not supported\n') 224 | 225 | if __name__ == '__main__': 226 | main = int(sys.version[0]) 227 | secondary = int(sys.version[2]) 228 | 229 | if main != 2 or secondary < 4: 230 | sys.exit('This program can not work (is not tested) with your python version (< 2.4 or >= 3.0)') 231 | 232 | if main == 2 and secondary == 4: 233 | version_warning() 234 | 235 | try: 236 | from pysnmp.smi import builder 237 | builder.MibBuilder().loadModules('SNMPv2-MIB', 'IF-MIB') 238 | except: 239 | sys.exit('This program requires python netsnmp\n> pip install pysnmp\n> pip install pysnmp_mibs') 240 | 241 | try: 242 | from pysnmp.proto.rfc1905 import NoSuchInstance 243 | except: 244 | # some version of pysnmp do not have this API :-( 245 | sys.exit( 246 | 'This program requires a version of pysnmp which is not compatible with the one installed\n' 247 | 'You _may_ be able to replace the installed version with the lastest one on pypi\n' 248 | '> pip install pysnmp\n' 249 | '> pip install pysnmp_mibs' 250 | ) 251 | 252 | from exaddos.configuration import ConfigurationError,load,ini,env,default 253 | 254 | next = '' 255 | arguments = { 256 | 'configuration' : '', 257 | } 258 | 259 | for arg in sys.argv[1:]: 260 | if next: 261 | arguments[next] = arg 262 | next = '' 263 | continue 264 | if arg in ['-c','--conf-file']: 265 | next = 'configuration' 266 | 267 | for arg in sys.argv[1:]: 268 | if arg in ['--',]: 269 | break 270 | if arg in ['-h','--help']: 271 | help() 272 | sys.exit(0) 273 | 274 | try: 275 | configuration = load(arguments['configuration']) 276 | except ConfigurationError,e: 277 | err('configuration issue, %s' % str(e)) 278 | sys.exit(1) 279 | 280 | for arg in sys.argv[1:]: 281 | if arg in ['--',]: 282 | break 283 | if arg in ['-h','--help']: 284 | help() 285 | sys.exit(0) 286 | if arg in ['-i','--ini']: 287 | ini() 288 | sys.exit(0) 289 | if arg in ['-e','--env']: 290 | env() 291 | sys.exit(0) 292 | if arg in ['-di','--diff-ini']: 293 | ini(True) 294 | sys.exit(0) 295 | if arg in ['-de','--diff-env']: 296 | env(True) 297 | sys.exit(0) 298 | if arg in ['-p','--pdb']: 299 | # The following may fail on old version of python (but is required for debug.py) 300 | os.environ['PDB'] = 'true' 301 | configuration.debug.pdb = True 302 | if arg in ['-D']: 303 | configuration.daemon.daemonize = True 304 | if arg in ['-m','--memory']: 305 | configuration.debug.memory = True 306 | 307 | # check the database is well only 400 by the user we use 308 | # start web server :) 309 | 310 | reactor.setup(configuration) 311 | 312 | if not drop_privileges(configuration): 313 | err('could not drop privileges') 314 | __exit(configuration.debug.memory,0) 315 | 316 | daemonise(configuration.daemon.daemonize) 317 | 318 | savepid(configuration.daemon.pidfile) 319 | 320 | if not configuration.profile.enable: 321 | try: 322 | reactor.run() 323 | except socket.error,e: 324 | # XXXX: Look at ExaBGP code fore better handling 325 | if e.errno == errno.EADDRINUSE: 326 | err('can not bind to %s:%d (port already/still in use)' % (configuration.http.host, configuration.http.port)) 327 | if e.errno == errno.EADDRNOTAVAIL: 328 | err('can not bind to %s:%d (IP unavailable)' % (configuration.http.host, configuration.http.port)) 329 | __exit(configuration.debug.memory,0) 330 | 331 | try: 332 | import cProfile as profile 333 | except: 334 | try: 335 | import profile 336 | except: 337 | err('could not perform profiling') 338 | class profile (object): 339 | @staticmethod 340 | def run (function): 341 | eval(function) 342 | 343 | profile.run('reactor.run()') 344 | __exit(configuration.debug.memory,0) 345 | -------------------------------------------------------------------------------- /lib/exaddos/configuration.py: -------------------------------------------------------------------------------- 1 | """ 2 | configuration.py 3 | 4 | Created by Thomas Mangin on 2014-02-07. 5 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 6 | """ 7 | 8 | # XXX: raised exception not caught 9 | # XXX: reloading mid-program not possible 10 | # XXX: validation for path, file, etc not correctly test (ie surely buggy) 11 | 12 | import os 13 | import sys 14 | import pwd 15 | import fnmatch 16 | 17 | class ConfigurationError (Exception): 18 | pass 19 | 20 | class NoneDict (dict): 21 | def __getitem__ (self,name): 22 | return None 23 | nonedict = NoneDict() 24 | 25 | class value (object): 26 | location = os.path.normpath(sys.argv[0]) if sys.argv[0].startswith('/') else os.path.normpath(os.path.join(os.getcwd(),sys.argv[0])) 27 | 28 | @staticmethod 29 | def integer (_): 30 | return int(_) 31 | 32 | # router only update SNMP counters every 10 seconds, any faster and all goes wrong 33 | @staticmethod 34 | def frequency (_): 35 | return int(_) if int(_) >= 10 else 10 36 | 37 | @staticmethod 38 | def lowunquote (_): 39 | return _.strip().strip('\'"').lower() 40 | 41 | @staticmethod 42 | def unquote (_): 43 | return _.strip().strip('\'"') 44 | 45 | @staticmethod 46 | def quote (_): 47 | return "'%s'" % str(_) 48 | 49 | @staticmethod 50 | def nop (_): 51 | return _ 52 | 53 | @staticmethod 54 | def boolean (_): 55 | return _.lower() in ('1','yes','on','enable','true') 56 | 57 | @staticmethod 58 | def methods (_): 59 | return _.upper().split() 60 | 61 | @staticmethod 62 | def list (_): 63 | return "'%s'" % ' '.join(_) 64 | 65 | @staticmethod 66 | def lower (_): 67 | return str(_).lower() 68 | 69 | @staticmethod 70 | def user (_): 71 | # XXX: incomplete 72 | try: 73 | pwd.getpwnam(_) 74 | # uid = answer[2] 75 | except KeyError: 76 | raise TypeError('user %s is not found on this system' % _) 77 | return _ 78 | 79 | @staticmethod 80 | def snmpauth (_): 81 | auth = value.unquote(_).upper() 82 | if auth not in ('','MD5','SHA','DES'): 83 | raise TypeError('SNMP authentication method unknown %s (expect MD5,SHA,DES)') 84 | return auth 85 | 86 | @staticmethod 87 | def snmppriv (_): 88 | priv = value.unquote(_).upper() 89 | if priv not in ('','DES','3DES','AES-128','AES-192','AES-256'): 90 | raise TypeError('SNMP authentication method unknown %s (expect 3DES,AES-128,AES-192,AES-256)') 91 | return priv 92 | 93 | @staticmethod 94 | def folder(path): 95 | path = os.path.expanduser(value.unquote(path)) 96 | paths = [ 97 | os.path.normpath(os.path.join(os.path.join(os.sep,*os.path.join(value.location.split(os.sep)[:-3])),path)), 98 | os.path.normpath(os.path.join('/','etc','exaddos','exaddos.conf',path)), 99 | os.path.normpath(path) 100 | ] 101 | options = [path for path in paths if os.path.exists(path)] 102 | if not options: raise TypeError('%s does not exists' % path) 103 | first = options[0] 104 | if not first: raise TypeError('%s does not exists' % first) 105 | return first 106 | 107 | @staticmethod 108 | def path (path): 109 | split = sys.argv[0].split('lib/exaddos') 110 | if len(split) > 1: 111 | prefix = os.sep.join(split[:1]) 112 | if prefix and path.startswith(prefix): 113 | path = path[len(prefix):] 114 | home = os.path.expanduser('~') 115 | if path.startswith(home): 116 | return "'~%s'" % path[len(home):] 117 | return "'%s'" % path 118 | 119 | @staticmethod 120 | def conf(path): 121 | first = value.folder(path) 122 | if not os.path.isfile(first): raise TypeError('%s is not a file' % path) 123 | return first 124 | 125 | @staticmethod 126 | def html(path): 127 | paths = [ 128 | os.path.normpath(os.path.join(os.path.join(os.sep,*os.path.join(value.location.split(os.sep)[:-3])),path)), 129 | os.path.normpath(os.path.join(os.path.join(os.sep,*os.path.join(value.location.split(os.sep)[:-3])),'data','exaddos','html')), 130 | os.path.normpath(os.path.join('/','var','lib','exaddos','html',path)), 131 | ] 132 | for folder in paths: 133 | if os.path.exists(folder) and os.path.isdir(folder): 134 | return folder 135 | raise TypeError('database could not be found') 136 | 137 | @staticmethod 138 | def database(path): 139 | paths = [ 140 | os.path.normpath(os.path.join(os.path.join(os.sep,*os.path.join(value.location.split(os.sep)[:-3])),path)), 141 | os.path.normpath(os.path.join(os.path.join(os.sep,*os.path.join(value.location.split(os.sep)[:-3])),'data','exaddos','db','exaddos.sqlite3')), 142 | os.path.normpath(os.path.join('/','var','lib','exaddos',path)), 143 | ] 144 | for database in paths: 145 | if os.path.exists(database): 146 | return database 147 | raise TypeError('database could not be found') 148 | 149 | @staticmethod 150 | def exe (path): 151 | first = value.conf(path) 152 | if not os.access(first, os.X_OK): raise TypeError('%s is not an executable' % first) 153 | return first 154 | 155 | @staticmethod 156 | def syslog (path): 157 | path = value.unquote(path) 158 | if path in ('stdout','stderr'): 159 | return path 160 | if path.startswith('host:'): 161 | return path 162 | return path 163 | 164 | defaults = { 165 | # 'database' : { 166 | # 'location' : (value.database,value.path, 'exaddos.sqlite3', 'the sqlite3 database location') 167 | # }, 168 | 'daemon' : { 169 | 'pidfile' : (value.unquote,value.quote, '', 'where to save the pid if we manage it'), 170 | 'user' : (value.user,value.quote, 'nobody', 'user to run as'), 171 | 'daemonize' : (value.boolean,value.lower, 'false', 'should we run in the background'), 172 | }, 173 | 174 | 'http' : { 175 | 'host' : (value.unquote,value.quote, '127.0.0.1', 'the tcp address the web server listens on'), 176 | 'port' : (value.integer,value.nop, '39200', 'port the web server listens on'), 177 | }, 178 | 179 | 'ipfix' : { 180 | 'host' : (value.unquote,value.quote, '127.0.0.1', 'the udp address the ipfix server listens on'), 181 | 'port' : (value.integer,value.nop, '29300 ', 'port the web server listens on'), 182 | }, 183 | 184 | 'location' : { 185 | 'html' : (value.html,value.quote, 'data/exaddos/html', 'the root of the html folder'), 186 | }, 187 | 188 | 'profile' : { 189 | 'enable' : (value.boolean,value.lower, 'false', 'enable profiling'), 190 | 'destination' : (value.syslog,value.quote, 'stdout', 'save profiling to file (instead of to the screen on exit)'), 191 | }, 192 | 193 | '[A-Z]*' : { 194 | 'router' : (value.unquote,value.quote, '127.0.0.1', 'the IP of the router to snmp poll (NO HOSTNAME)'), 195 | 'snmp_version' : (value.integer,value.nop, '2', 'only version 2 supported'), 196 | 'snmp_password' : (value.unquote,value.quote, 'public', 'your passwords are secure aren\'t they'), 197 | 'snmp_user' : (value.unquote,value.quote, '', 'snmp v3 user'), 198 | 'snmp_auth_key' : (value.unquote,value.quote, '', 'snmp v3 auth key'), 199 | 'snmp_auth_method' : (value.snmpauth,value.quote, '', 'snmp v3 auth (MD5,SHA,DES, empty string for no auth)'), 200 | 'snmp_privacy_key' : (value.unquote,value.quote, '', 'snmp v3 privacy key'), 201 | 'snmp_privacy_method' : (value.snmppriv,value.quote, '', 'snmp v3 privacy (DES,3DES,AES-128,AES-192,AES-256, empty for no privacy)'), 202 | 'snmp_frequency' : (value.frequency,value.nop, '10', 'snmp pulling frequency (minimum 10 seconds)'), 203 | 'snmp_index_port' : (value.integer,value.nop, '0', 'physical interface SNMP interface index'), 204 | 'snmp_index_vlan' : (value.integer,value.nop, '0', 'vlan/ae/other SNMP interface index (or physical if not defined)'), 205 | 'threshold_bandwidth' : (value.integer,value.nop, '0', 'threshold for abnormal unicast traffic (bits)'), 206 | 'threshold_unicast' : (value.integer,value.nop, '0', 'threshold for abnormal unicast traffic (pps)'), 207 | 'threshold_notunicast' : (value.integer,value.nop, '0', 'threshold for abnormal non unicast (icmp mostly) traffic (pps)'), 208 | }, 209 | 210 | # Here for internal use 211 | 'internal' : { 212 | 'name' : (value.nop,value.nop, 'ExaDDOS', 'name'), 213 | 'version' : (value.nop,value.nop, '0.1.0', 'version'), 214 | }, 215 | 216 | # Here for internal use 217 | 'debug' : { 218 | 'pdb' : (value.boolean,value.lower,'false','command line option --pdb'), 219 | 'memory' : (value.boolean,value.lower,'false','command line option --memory'), 220 | }, 221 | } 222 | 223 | import ConfigParser 224 | 225 | class Store (dict): 226 | def __getitem__ (self,key): 227 | return dict.__getitem__(self,key.replace('_','-')) 228 | def __setitem__ (self,key,value): 229 | return dict.__setitem__(self,key.replace('_','-'),value) 230 | def __getattr__ (self,key): 231 | return dict.__getitem__(self,key.replace('_','-')) 232 | def __setattr__ (self,key,value): 233 | return dict.__setitem__(self,key.replace('_','-'),value) 234 | 235 | 236 | def _configuration (conf): 237 | location = os.path.join(os.sep,*os.path.join(value.location.split(os.sep))) 238 | while location: 239 | location, directory = os.path.split(location) 240 | if directory == 'lib': 241 | break 242 | 243 | _conf_paths = [] 244 | if conf: 245 | _conf_paths.append(os.path.abspath(os.path.normpath(conf))) 246 | if location: 247 | _conf_paths.append(os.path.normpath(os.path.join(location,'etc','exaddos','exaddos.conf'))) 248 | _conf_paths.append(os.path.normpath(os.path.join('/','etc','exaddos','exaddos.conf'))) 249 | 250 | try: 251 | ini_file = [path for path in _conf_paths if os.path.exists(path)][0] 252 | except IndexError: 253 | ini_file = None 254 | 255 | if not ini_file: 256 | raise ConfigurationError('could not find exaddos configuration file') 257 | 258 | ini = ConfigParser.ConfigParser() 259 | ini.read(ini_file) 260 | 261 | templates = {} 262 | for section in defaults: 263 | for wildcard in ('*','?','['): 264 | if wildcard in section: 265 | templates[section] = defaults[section] 266 | 267 | for section in templates: 268 | del defaults[section] 269 | 270 | for section in ini.sections(): 271 | for template in templates: 272 | search = 'exaddos.%s' % template 273 | if fnmatch.fnmatch(section,search): 274 | defaults[section[len('exaddos.'):]] = templates[template] 275 | 276 | configuration = Store() 277 | 278 | for section in defaults: 279 | default = defaults[section] 280 | 281 | for option in default: 282 | convert = default[option][0] 283 | try: 284 | proxy_section = 'exaddos.%s' % section 285 | env_name = '%s.%s' % (proxy_section,option) 286 | conf = value.unquote(os.environ.get(env_name,'')) \ 287 | or value.unquote(os.environ.get(env_name.replace('.','_'),'')) \ 288 | or value.unquote(ini.get(proxy_section,option,nonedict)) \ 289 | or default[option][2] 290 | except (ConfigParser.NoSectionError,ConfigParser.NoOptionError): 291 | conf = default[option][2] 292 | try: 293 | configuration.setdefault(section,Store())[option] = convert(conf) 294 | except TypeError: 295 | raise ConfigurationError('invalid value for %s.%s : %s' % (section,option,conf)) 296 | 297 | return configuration 298 | 299 | __configuration = None 300 | 301 | def load (conf=None): 302 | global __configuration 303 | if __configuration: 304 | return __configuration 305 | if conf is None: 306 | raise RuntimeError('You can not have an import using load() before main() initialised it') 307 | __configuration = _configuration(conf) 308 | return __configuration 309 | 310 | def default (): 311 | for section in sorted(defaults): 312 | if section in ('internal','debug'): 313 | continue 314 | for option in sorted(defaults[section]): 315 | values = defaults[section][option] 316 | default = "'%s'" % values[2] if values[1] in (value.list,value.path,value.quote,value.syslog) else values[2] 317 | yield 'exaddos.%s.%s %s: %s. default (%s)' % (section,option,' '*(27-len(section)-len(option)),values[3],default) 318 | 319 | def ini (diff=False): 320 | for section in sorted(__configuration): 321 | if section in ('internal','debug'): 322 | continue 323 | header = '\n[exaddos.%s]' % section 324 | for k in sorted(__configuration[section]): 325 | v = __configuration[section][k] 326 | if diff and defaults[section][k][0](defaults[section][k][2]) == v: 327 | continue 328 | if header: 329 | print header 330 | header = '' 331 | print '%s = %s' % (k,defaults[section][k][1](v)) 332 | 333 | def env (diff=False): 334 | print 335 | for section,values in __configuration.items(): 336 | if section in ('internal','debug'): 337 | continue 338 | for k,v in values.items(): 339 | if diff and defaults[section][k][0](defaults[section][k][2]) == v: 340 | continue 341 | if defaults[section][k][1] == value.quote: 342 | print "exaddos.%s.%s='%s'" % (section,k,v) 343 | continue 344 | print "exaddos.%s.%s=%s" % (section,k,defaults[section][k][1](v)) 345 | -------------------------------------------------------------------------------- /lib/exaddos/container.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | container.py 4 | 5 | Created by Thomas Mangin on 2014-02-06. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | # Ultimately these classes could use their own thread, a queue for local storage and redis as an option to allow multiple collection points 10 | 11 | from threading import Lock 12 | from copy import deepcopy 13 | from time import time 14 | 15 | class ContainerSNMP (object): 16 | 17 | def __init__ (self,max_speakers=5): 18 | self._lock = Lock() 19 | self._data = {} 20 | 21 | def keys (self): 22 | with self._lock: 23 | return self._data.keys() 24 | 25 | def set (self,name,d): 26 | with self._lock: 27 | r = self._data.setdefault(name,{}) 28 | for k,v in d.iteritems(): 29 | r[k] = v 30 | 31 | def get (self,name): 32 | with self._lock: 33 | return dict(self._data[name].iteritems()) 34 | 35 | def data (self): 36 | with self._lock: 37 | return deepcopy(self._data) 38 | 39 | 40 | class ContainerFlow (object): 41 | retain = 60 42 | #retain = 3600 # XXX: for debugging 43 | 44 | def __init__ (self,max_speakers=5): 45 | self._traffic_lock = Lock() 46 | 47 | # flow code 48 | self._counters = {} 49 | self._overall = {} 50 | self._threshold = {} 51 | self._traffic = {} 52 | 53 | self._monitor_lock = Lock() 54 | self._monitor = {} 55 | self._monitored = {} 56 | 57 | self._max_speaker = max_speakers 58 | self.period = 5 59 | 60 | for minute in range(0,self.period): 61 | self.make_minute(minute) 62 | 63 | def make_minute (self,minute): 64 | counter = self._counters 65 | 66 | default = { 67 | 'sipv4' : (127 << 24) + 1, 68 | 'dipv4' : (127 << 24) + 1, 69 | 'sport' : 0, 70 | 'dport' : 0, 71 | } 72 | 73 | if minute not in counter: 74 | counter[minute] = {} 75 | for direction in ('sipv4','dipv4','sport','dport'): 76 | for counter in ('bytes','pckts','flows'): 77 | # numbers need to be unique, and lower than our traffic 78 | self._threshold.setdefault(minute,{}).setdefault(direction,{})[counter] = list(range(-1,-self._max_speaker-1,-1)) 79 | self._traffic.setdefault(minute,{}).setdefault(direction,{})[counter] = dict(zip(zip(range(-1,-self._max_speaker-1,-1),[minute,]*self._max_speaker),[default[direction],]*self._max_speaker)) 80 | 81 | def purge_minute (self,minute): 82 | counter = self._counters 83 | for past in self._threshold.keys()[:-self.period]: 84 | del counter[past] 85 | del self._threshold[past] 86 | del self._traffic[past] 87 | 88 | def ipfix (self,update): 89 | minute = int(update['epoch'])/60 90 | 91 | with self._traffic_lock: 92 | self.purge_minute(minute) 93 | self.make_minute(minute) 94 | 95 | overall = self._overall 96 | counter = self._counters 97 | 98 | proto = update['proto'] 99 | sipv4 = update['sipv4'] 100 | dipv4 = update['dipv4'] 101 | bytes = update['bytes'] 102 | pckts = update['pckts'] 103 | flows = update['flows'] 104 | 105 | counter_total = overall.setdefault('total',{}) 106 | counter_total['bytes'] = counter_total.get('bytes',0) + bytes 107 | counter_total['pckts'] = counter_total.get('pckts',0) + pckts 108 | counter_total['flows'] = counter_total.get('flows',0) + flows 109 | 110 | if update['proto'] == 6: # TCP 111 | counter_tcp = overall.setdefault('tcp',{}) 112 | counter_tcp['bytes'] = counter_tcp.get('bytes',0) + bytes 113 | counter_tcp['pckts'] = counter_tcp.get('pckts',0) + pckts 114 | counter_tcp['flows'] = counter_tcp.get('flows',0) + flows 115 | elif update['proto'] == 17: # UDP 116 | counter_udp = overall.setdefault('udp',{}) 117 | counter_udp['bytes'] = counter_udp.get('bytes',0) + bytes 118 | counter_udp['pckts'] = counter_udp.get('pckts',0) + pckts 119 | counter_udp['flows'] = counter_udp.get('flows',0) + flows 120 | else: 121 | counter_other = overall.setdefault('other',{}) 122 | counter_other['bytes'] = counter_other.get('bytes',0) + bytes 123 | counter_other['pckts'] = counter_other.get('pckts',0) + pckts 124 | counter_other['flows'] = counter_other.get('flows',0) + flows 125 | 126 | #sipv4[time]['sipv4'/'dipv4'/'proto'][sipv4 ip]['pckts'/'bytes'/'flows'] 127 | counter_sipv4 = counter.setdefault(minute,{}).setdefault('sipv4',{}).setdefault(sipv4,{'pckts': 0, 'bytes': 0, 'flows':0}) 128 | counter_sipv4['bytes'] += bytes 129 | counter_sipv4['pckts'] += pckts 130 | counter_sipv4['flows'] += flows 131 | 132 | counter_dipv4 = counter.setdefault(minute,{}).setdefault('dipv4',{}).setdefault(dipv4,{'pckts': 0, 'bytes': 0, 'flows':0}) 133 | counter_dipv4['bytes'] += bytes 134 | counter_dipv4['pckts'] += pckts 135 | counter_dipv4['flows'] += flows 136 | 137 | counter_proto = counter.setdefault(minute,{}).setdefault('proto',{}).setdefault(proto,{'pckts': 0, 'bytes': 0, 'flows':0}) 138 | counter_proto['bytes'] += bytes 139 | counter_proto['pckts'] += pckts 140 | counter_proto['flows'] += flows 141 | 142 | if 'sport' in update: 143 | counter_sport = counter.setdefault(minute,{}).setdefault('sport',{}).setdefault(update['sport'],{'pckts': 0, 'bytes': 0, 'flows':0}) 144 | counter_sport['bytes'] += bytes 145 | counter_sport['pckts'] += pckts 146 | counter_sport['flows'] += flows 147 | else: 148 | counter_sport = None 149 | 150 | if 'dport' in update: 151 | counter_dport = counter.setdefault(minute,{}).setdefault('dport',{}).setdefault(update['dport'],{'pckts': 0, 'bytes': 0, 'flows':0}) 152 | counter_dport['bytes'] += bytes 153 | counter_dport['pckts'] += pckts 154 | counter_dport['flows'] += flows 155 | else: 156 | counter_dport = None 157 | 158 | for direction,data in (('sipv4',counter_sipv4),('dipv4',counter_dipv4),('sport',counter_sport),('dport',counter_dport)): 159 | for counter in ('bytes','pckts','flows'): 160 | traffic = self._traffic[minute][direction][counter] 161 | maximum = self._threshold[minute][direction][counter] 162 | ip = update[direction] 163 | value = data[counter] 164 | drop = maximum[0] 165 | 166 | if value > drop: 167 | traffic_items = list(traffic.iteritems()) 168 | traffic_values, traffic_ips = zip(*traffic_items) 169 | traffic_number,traffic_minutes = zip(*traffic_values) 170 | 171 | # updating the number of seen for a ip 172 | if ip in traffic_ips: 173 | # key is a (value,time) tuple 174 | for key,host in traffic_items: 175 | if ip == host: 176 | traffic[key] = ip 177 | maximum = sorted(maximum + [value,]) 178 | break 179 | # replacing an entry with a new value 180 | else: 181 | maximum = sorted(maximum[1:] + [value,]) 182 | self._threshold[minute][direction][counter] = maximum 183 | 184 | for index,host in traffic_items: 185 | if ip == host: break 186 | 187 | timestamp = minute 188 | # prevent duplicate entries using a unique timestamp 189 | while timestamp in traffic_minutes: 190 | timestamp += 1 191 | 192 | # del traffic[drop] 193 | del self._traffic[minute][direction][counter][index] 194 | traffic[(value,timestamp)] = ip 195 | 196 | record = self.monitored() 197 | if sipv4 in record: 198 | self.monitor_record(sipv4,sipv4,dipv4,proto,bytes,pckts,flows,update.get('sport',-1),update.get('dport',-1)) 199 | elif dipv4 in record: 200 | self.monitor_record(dipv4,sipv4,dipv4,proto,bytes,pckts,flows,update.get('sport',-1),update.get('dport',-1)) 201 | 202 | def overall (self): 203 | with self._traffic_lock: 204 | return deepcopy(self._overall) 205 | 206 | def traffic (self): 207 | with self._traffic_lock: 208 | return deepcopy(self._traffic) 209 | 210 | def monitor (self,ipn): 211 | with self._monitor_lock: 212 | self._monitor[ipn] = time() 213 | 214 | def monitored (self): 215 | now = time() 216 | r = [] 217 | with self._monitor_lock: 218 | for ipn,back in list(self._monitor.iteritems()): 219 | # remember seeing the ip for 60 seconds 220 | if back + self.retain < now: 221 | del self._monitor[ipn] 222 | if ipn in self._monitored: 223 | del self._monitored[ipn] 224 | continue 225 | r.append(ipn) 226 | return r 227 | 228 | def monitor_record (self,ipn,sipv4,dipv4,proto,bytes,pckts,flows,sport,dport): 229 | # JSON does not let us have list has key :( 230 | if sport > 0 and dport > 0: 231 | flow = "%d %d %d %d" % (sipv4,sport,dipv4,dport) 232 | else: 233 | flow = "%d %d" % (sipv4,dipv4) 234 | 235 | with self._monitor_lock: 236 | values = self._monitored.setdefault(ipn,{}).setdefault(proto,{}).setdefault(flow,{}) 237 | values['pckts'] = values.get('pckts',0) + pckts 238 | values['bytes'] = values.get('bytes',0) + bytes 239 | values['flows'] = values.get('flows',0) + flows 240 | 241 | def monitor_data (self,ipn): 242 | with self._monitor_lock: 243 | return deepcopy(self._monitored.get(ipn,{})) 244 | -------------------------------------------------------------------------------- /lib/exaddos/debug.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | debug.py 4 | 5 | Created by Thomas Mangin on 2014-02-06. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import os 10 | import sys 11 | 12 | import traceback 13 | 14 | def bug_report (type, value, trace): 15 | print >> sys.stderr, '' 16 | print >> sys.stderr, '' 17 | print >> sys.stderr, '-'*80 18 | print >> sys.stderr, '-- Please provide the information below: https://github.com/Exa-Networks/exaddos' 19 | print >> sys.stderr, '-'*80 20 | print >> sys.stderr, '' 21 | print >> sys.stderr, '' 22 | print >> sys.stderr, '-- Version' 23 | print >> sys.stderr, '' 24 | print >> sys.stderr, sys.version 25 | print >> sys.stderr, '' 26 | print >> sys.stderr, '' 27 | print >> sys.stderr, '-- Traceback' 28 | print >> sys.stderr, '' 29 | print >> sys.stderr, '' 30 | traceback.print_exception(type,value,trace) 31 | print >> sys.stderr, '' 32 | print >> sys.stderr, '' 33 | print >> sys.stderr, '-'*80 34 | print >> sys.stderr, '-- Please provide the information above: https://github.com/Exa-Networks/exaddos' 35 | print >> sys.stderr, '-'*80 36 | print >> sys.stderr, '' 37 | print >> sys.stderr, '' 38 | 39 | #print >> sys.stderr, 'the program failed with message :', value 40 | 41 | def intercept (type, value, trace): 42 | interactive = os.environ.get('PDB',None) 43 | 44 | if interactive in ['0','']: 45 | # PDB was set to 0 or '' which is undocumented, and we do nothing 46 | pass 47 | else: 48 | bug_report(type, value, trace) 49 | if interactive == 'true': 50 | import pdb 51 | pdb.pm() 52 | 53 | sys.excepthook = intercept 54 | 55 | if sys.argv: 56 | del sys.argv[0] 57 | __file__ = os.path.abspath(sys.argv[0]) 58 | __name__ = '__main__' 59 | execfile(sys.argv[0]) 60 | -------------------------------------------------------------------------------- /lib/exaddos/flow.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | flow.py 4 | 5 | Created by Thomas Mangin on 2014-02-09. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import sys 10 | import socket 11 | import time 12 | 13 | from .thread import Thread 14 | from .ipfix import IPFIX 15 | from .q import Queue,Empty 16 | from .log import log,err 17 | 18 | class _FlowServerConsumer (object): 19 | use_thread = True 20 | 21 | def __init__ (self,queue,container,raising): 22 | self.consumerd = None 23 | self.queue = queue 24 | self.container = container 25 | self.raising = raising 26 | self.parser = IPFIX(container.ipfix) 27 | self.running = True 28 | 29 | def serve (self): 30 | last = time.time() - 10 31 | if self.use_thread: 32 | try: 33 | while self.running: 34 | try: 35 | while True: 36 | size = self.queue.qsize() 37 | if size > 1000: 38 | now = time.time() 39 | if now - last > 1: 40 | err('warning, ipfix data is generated faster than we can consumme, %d messages queued' % size) 41 | last = now 42 | data = self.queue.get() 43 | self.parser.read(data) 44 | except Empty: 45 | pass 46 | except Exception,e: 47 | self.running = False 48 | raise e 49 | else: 50 | # debug without starting a thread 51 | while self.running: 52 | try: 53 | while True: 54 | data = self.queue.get() 55 | self.parser.read(data) 56 | except Empty: 57 | pass 58 | 59 | def start (self): 60 | log('starting ipfix consummer') 61 | if self.use_thread: 62 | self.consumerd = Thread(self.serve,self.raising) 63 | self.consumerd.daemon = True 64 | self.consumerd.start() 65 | else: 66 | self.serve() 67 | 68 | def alive (self): 69 | return self.running 70 | 71 | def join (self): 72 | if self.consumerd: 73 | self.consumerd.join(0.1) 74 | 75 | class _FlowServerFactory (object): 76 | use_thread = True 77 | 78 | def __init__ (self,host,port,queue,raising): 79 | self.host = host 80 | self.port = port 81 | self.queue = queue 82 | self.raising = Queue() 83 | self.running = True 84 | 85 | def serve (self): 86 | # The best a 10 Mb interface can do is 14,880 frames per second 87 | # each frame being only 84 bytes 88 | # On 127.0.0.1, my Mac can receive around 10/12k frames .. 89 | # So it would seems the MAC loopback is bad for high perf networking ... 90 | try: 91 | sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) 92 | current = sock.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF) # value of 196724 on my mac 93 | new = current 94 | while True: 95 | try: 96 | new += current 97 | sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, new) 98 | except socket.error: 99 | log('ipfix changed SO_RCVBUF from %d to %d' % (current,new-current)) 100 | sys.stdout.flush() 101 | break 102 | 103 | sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) 104 | sock.bind((self.host,self.port)) 105 | self.running = True 106 | except: 107 | err('could not start ipfix server') 108 | raise 109 | 110 | if self.use_thread: 111 | try: 112 | while self.running: 113 | data, addr = sock.recvfrom(8192) 114 | data = self.queue.put(data) 115 | except Exception,e: 116 | self.running = False 117 | raise e 118 | else: 119 | # debug without starting a thread 120 | while self.running: 121 | data, addr = sock.recvfrom(8192) 122 | data = self.queue.put(data) 123 | 124 | def start (self): 125 | log('starting ipfix server') 126 | if self.use_thread: 127 | self.flowd = Thread(self.serve,self.raising) 128 | self.flowd.daemon = True 129 | self.flowd.start() 130 | else: 131 | self.serve() 132 | 133 | def alive (self): 134 | return self.running 135 | 136 | def join (self): 137 | if self.flowd: 138 | self.flowd.join(0.1) 139 | 140 | 141 | class FlowServer (object): 142 | servers = {} 143 | consumer = {} 144 | 145 | def __init__ (self,configuration,container): 146 | # This will be shared among all instrance 147 | self.configuration = configuration 148 | self.container = container 149 | 150 | def add (self,host,port,raising): 151 | queue = Queue() 152 | 153 | key = '%s:%d' % (host,port) 154 | if key not in self.servers: 155 | # start 3 consumers per ipfix server 156 | for index in range(3): 157 | consumer = _FlowServerConsumer(queue,self.container,raising) 158 | consumer.parent = self 159 | self.consumer[key+str(index)] = consumer 160 | 161 | server = _FlowServerFactory(host,port,queue,raising) 162 | server.parent = self 163 | self.servers[key] = server 164 | 165 | def run (self,daemon): 166 | for key in self.consumer: 167 | if daemon and self.consumer[key].use_thread: 168 | self.consumer[key].start() 169 | if not daemon and not self.consumer[key].use_thread: 170 | self.consumer[key].start() 171 | 172 | for key in self.servers: 173 | if daemon and self.servers[key].use_thread: 174 | self.servers[key].start() 175 | if not daemon and not self.servers[key].use_thread: 176 | self.servers[key].start() 177 | 178 | def join (self): 179 | for key in self.servers: 180 | self.servers[key].join() 181 | 182 | def alive (self): 183 | for key in self.servers: 184 | if self.servers[key].flowd.isAlive(): 185 | return True 186 | return False 187 | -------------------------------------------------------------------------------- /lib/exaddos/http.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | http.py 4 | 5 | Created by Thomas Mangin on 2014-02-06. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import os 10 | import sys 11 | import urlparse 12 | import SimpleHTTPServer 13 | import SocketServer 14 | import json 15 | import socket 16 | 17 | from .log import log,err 18 | 19 | # ugly but practical for testing .. 20 | if __name__ != '__main__': 21 | from .thread import Thread 22 | 23 | def nop (arg): 24 | return arg 25 | 26 | # def snmp_json (data): 27 | # s = {} 28 | # for link,information in data.iteritems(): 29 | # s.setdefault(information['ifHCInUcastPkts'] + information['ifInNUcastPkts'],[]).append(link) 30 | 31 | # display = sorted(s.keys()) 32 | # display.reverse() 33 | # r = [] 34 | # for speed in display: 35 | # links = s[speed] 36 | # for link in sorted(links): 37 | # d = {'link':link} 38 | # information = data[link] 39 | # for k,v in information.iteritems(): 40 | # d[k] = v 41 | # r.append(d) 42 | # return json.dumps(r) 43 | 44 | # def flow_overall (data): 45 | # r = {} 46 | # for proto in data: 47 | # for counter in data[proto]: 48 | # r["%s_%s" % (proto,counter)] = data[proto][counter] 49 | # return json.dumps(r) 50 | 51 | 52 | def flow_traffic (data): 53 | nb_keeping = 5 54 | 55 | default = { 56 | 'sipv4' : "127.0.0.1", 57 | 'dipv4' : "127.0.0.1", 58 | 'sport' : "0", 59 | 'dport' : "0", 60 | } 61 | 62 | best = {} 63 | maximum = {} 64 | for direction in ('sipv4','dipv4','sport','dport'): 65 | for counter in ('bytes','pckts','flows'): 66 | best['%s_%s' % (direction,counter)] = {-1:[default[direction],]} 67 | maximum['%s_%s' % (direction,counter)] = [-1,] * nb_keeping 68 | 69 | for t in data: 70 | for d in data[t]: 71 | for c in data[t][d]: 72 | info = data[t][d][c] 73 | index = '%s_%s' % (d,c) 74 | for key in info: 75 | number,minute = key 76 | if number >= maximum[index][0]: 77 | best[index].setdefault(number,[]).append(info[key]) 78 | maximum[index] = sorted(maximum[index][1:]+[number,]) 79 | 80 | # from pprint import pprint 81 | # print 82 | # pprint(best) 83 | # print 84 | 85 | r = {} 86 | for direction in ('sipv4','dipv4','sport','dport'): 87 | r[direction] = {} 88 | for counter in ('bytes','pckts','flows'): 89 | index = '%s_%s' % (direction,counter) 90 | v = {} 91 | 92 | for number in reversed(maximum[index]): 93 | for key in set(best[index][number]): 94 | v[key] = v.get(key,0) + number 95 | 96 | l = [] 97 | for key,value in v.iteritems(): 98 | if value > 0: 99 | l.append({'key': key, 'value': str(value)}) 100 | 101 | # XXX: This is really some code to fix the HTML 102 | for _ in range(max(0,nb_keeping - len(l))): 103 | l.append({'key': '', 'value' : ''}) 104 | 105 | r[direction][counter] = l[:nb_keeping] 106 | 107 | return json.dumps(r) 108 | 109 | def json_index (data): 110 | return """ 111 | 112 | 113 | %s 114 | 115 | 116 | """ % '
\n'.join(['%s' % (path,path,path) for path in data]) 117 | 118 | class HTTPHandler (SimpleHTTPServer.SimpleHTTPRequestHandler): 119 | # webroot is added to this class 120 | # snmp is added to this class 121 | # flow is added to this class 122 | 123 | # monkey patching 3.3 fix http://hg.python.org/cpython/rev/7e5d7ef4634d 124 | def finish (self): 125 | try: 126 | SimpleHTTPServer.SimpleHTTPRequestHandler.finish(self) 127 | except socket.error: 128 | # should really check it is really ECONNABORTED 129 | pass 130 | 131 | def __init__ (self,*args,**kargs): 132 | self._json = { 133 | "/json.html" : ( 'text/html', json_index, self.json_list, () ), 134 | "/json/" : ( 'text/html', json_index, self.json_list, () ), 135 | "/json/snmp/interfaces.json" : ( 'text/json', json.dumps, self.snmp.data, () ), 136 | "/json/flow/overview.json" : ( 'text/json', json.dumps, self.flow.overall, () ), 137 | "/json/flow/talkers.json" : ( 'text/json', flow_traffic, self.flow.traffic, () ), 138 | } 139 | try: 140 | SimpleHTTPServer.SimpleHTTPRequestHandler.__init__(self,*args,**kargs) 141 | except socket.error: 142 | # should really check it is really ECONNABORTED 143 | pass 144 | 145 | def json_list (self): 146 | r = [] 147 | for name in self._json: 148 | r.append(name) 149 | return r 150 | 151 | def log_message (*args): 152 | pass 153 | 154 | def valid_path (self,path): 155 | for letter in path: 156 | if letter.isalnum(): 157 | continue 158 | if letter in ('-','_','.','/'): 159 | continue 160 | return False 161 | 162 | return '..' not in path 163 | 164 | def do_HEAD(self): 165 | self.send_response(200) 166 | self.send_header('Content-type', 'text/html') 167 | self.end_headers() 168 | return 169 | 170 | def do_GET (self): 171 | content = '' 172 | fname = '' 173 | 174 | # Parse query data to find out what was requested 175 | parsedParams = urlparse.urlparse(self.path) 176 | path = parsedParams.path 177 | 178 | if path.startswith('/json/'): 179 | code = 200 180 | 181 | if path in self._json: 182 | encoding, presentation, source, param = self._json[path] 183 | content = presentation(source(),*param) 184 | else: 185 | code = 404 186 | encoding = 'text/html' 187 | content = '404' 188 | 189 | if code == 404: 190 | err('http server could not serve json %s' % path) 191 | 192 | elif self.valid_path(path): 193 | if path == '/': 194 | path = '/index.html' 195 | code = 200 196 | if path.endswith('.js'): 197 | encoding = 'application/x-javascript' 198 | elif path.endswith('.html'): 199 | encoding = 'text/html' 200 | elif path.endswith('.css'): 201 | encoding = 'text/css' 202 | elif path.endswith(('.jpg','.jpeg')): 203 | encoding = 'image/jpeg' 204 | elif path.endswith('.png'): 205 | encoding = 'image/png' 206 | elif path.endswith('.gif'): 207 | encoding = 'image/gif' 208 | else: 209 | encoding = 'text/plain' 210 | fname = os.path.join(self.webroot,path.lstrip('/')) 211 | 212 | if fname and os.path.isfile(fname): 213 | try: 214 | with open(fname,'r') as f: 215 | content = f.read() 216 | except Exception: 217 | code = 500 218 | encoding = 'text/html' 219 | content = 'could not read the file' 220 | else: 221 | code = 404 222 | encoding = 'text/html' 223 | content = '404' 224 | 225 | if code == 404: 226 | err('http server could not serve path %s -> %s' % (path, fname)) 227 | 228 | else: 229 | code = 404 230 | encoding = 'text/html' 231 | content = '404' 232 | 233 | self.send_response(code) 234 | self.send_header('Content-type', encoding) 235 | self.end_headers() 236 | self.wfile.write(content) 237 | 238 | return 239 | 240 | def do_POST (self): 241 | # Parse query data to find out what was requested 242 | parsedParams = urlparse.urlparse(self.path) 243 | path = parsedParams.path 244 | 245 | if path == '/post/monitor/ip': 246 | import cgi 247 | form = cgi.FieldStorage( 248 | fp=self.rfile, 249 | headers=self.headers, 250 | environ={'REQUEST_METHOD':'POST', 251 | 'CONTENT_TYPE':self.headers['Content-Type'], 252 | }) 253 | 254 | ip = form.getfirst('ip','') 255 | number = long(ip) if ip and ip.isdigit() else 0 256 | 257 | if number and number < (224 << 24) - 1: # last unicast IP 258 | self.flow.monitor(number) 259 | self.send_response(200) 260 | self.send_header('Content-type','text/json') 261 | self.end_headers() 262 | self.wfile.write(json.dumps(self.flow.monitor_data(number))) 263 | return 264 | 265 | self.send_response(404) 266 | self.end_headers() 267 | 268 | 269 | class _HTTPServerFactory (object): 270 | use_thread = True 271 | 272 | def __init__ (self,host,port,raising): 273 | log('http server on %s:%d' % (host,port)) 274 | self.httpd = None 275 | self.raising = raising 276 | 277 | self.host = host 278 | self.port = port 279 | 280 | def serve (self): 281 | SocketServer.TCPServer.allow_reuse_address = True 282 | server = SocketServer.TCPServer((self.host, self.port),HTTPHandler) 283 | server.serve_forever() 284 | 285 | def start (self): 286 | log('starting http server') 287 | if self.use_thread: 288 | self.httpd = Thread(self.serve,self.raising) 289 | self.httpd.daemon = True 290 | self.httpd.start() 291 | else: 292 | self.serve() 293 | 294 | def join (self): 295 | if self.httpd: 296 | self.httpd.join(0.1) 297 | 298 | 299 | class HTTPServer (object): 300 | servers = {} 301 | 302 | def __init__ (self,configuration,snmp,flow): 303 | HTTPHandler.webroot = configuration.location.html 304 | # This will be shared among all instrance 305 | HTTPHandler.snmp = snmp 306 | HTTPHandler.flow = flow 307 | 308 | def add (self,host,port,raising): 309 | key = '%s:%d' % (host,port) 310 | if key not in self.servers: 311 | server = _HTTPServerFactory(host,port,raising) 312 | server.parent = self 313 | self.servers[key] = server 314 | 315 | def run (self,daemon): 316 | for key in self.servers: 317 | if daemon and self.servers[key].use_thread: 318 | self.servers[key].start() 319 | if not daemon and not self.servers[key].use_thread: 320 | self.servers[key].start() 321 | 322 | def join (self): 323 | for key in self.servers: 324 | self.servers[key].join() 325 | 326 | def alive (self): 327 | for key in self.servers: 328 | if self.servers[key].httpd.isAlive(): 329 | return True 330 | return False 331 | 332 | 333 | if __name__ == '__main__': 334 | one={23200748: {'dipv4': {'pckts': {(5300, 23200749): 1490932248, (-1, 23200748): 2130706433, (103, 23200750): 1390085485, (107, 23200753): 1390092013, (19561, 23200752): 1490932231}, 'bytes': {(720, 23200753): 1390129370, (-1, 23200748): 2130706433, (56647, 23200750): 1490932238, (409776, 23200752): 1390085475, (28836876, 23200749): 1490932231}, 'flows': {(11, 23200752): 1490932234, (-1, 23200748): 2130706433, (7, 23200750): 1390085151, (43, 23200753): 1490932240, (69, 23200749): 1390085144}}, 'sipv4': {'pckts': {(725, 23200750): 1390093537, (-1, 23200748): 2130706433, (43, 23200753): 3331355530, (15664, 23200752): 1390092013, (7, 23200751): 1490932229}, 'bytes': {(58138, 23200751): 1490932242, (-1, 23200748): 2130706433, (6312366, 23200753): 1490932234, (890, 23200750): 3163192613, (2978981, 23200752): 2439115988}, 'flows': {(16, 23200753): 1390085144, (-1, 23200748): 2130706433, (7, 23200750): 1490932239, (15, 23200749): 1390085150, (35, 23200751): 134744072}}}, 4: {'dipv4': {'pckts': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'bytes': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'flows': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}}, 'sipv4': {'pckts': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'bytes': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'flows': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}}}, 23200749: {'dipv4': {'pckts': {(12565, 23200750): 1490932239, (9, 23200749): 1490932241, (281, 23200752): 1490932237, (6, 23200754): 1490932233, (9596, 23200751): 1490932229}, 'bytes': {(18710183, 23200750): 1490932239, (10659, 23200751): 1490932243, (3795, 23200754): 1159668081, (415976, 23200749): 1390085658, (14105, 23200753): 1490932248}, 'flows': {(17, 23200754): 1490932243, (3, 23200751): 1490932234, (18, 23200749): 1490932233, (43, 23200753): 1490932249, (89, 23200752): 1490932242}}, 'sipv4': {'pckts': {(41, 23200749): 520834244, (391, 23200750): 1390087841, (4747, 23200753): 1249759509, (14, 23200752): 1490932238, (7975, 23200754): 34911506}, 'bytes': {(10659, 23200751): 3245197261, (546951, 23200750): 1390087841, (311439, 23200749): 1490932245, (613, 23200752): 520838772, (11954818, 23200754): 34911506}, 'flows': {(3, 23200751): 1490932245, (16, 23200752): 1390085144, (8, 23200749): 1490932238, (65, 23200750): 1490932239, (2, 23200754): 1490932240}}}, 23200750: {'dipv4': {'pckts': {(4541, 23200753): 1490932237, (1081, 23200752): 35009072, (4610, 23200750): 1490932231, (157, 23200754): 1490932233, (7, 23200751): 1490932248}, 'bytes': {(92806, 23200755): 1490932242, (11062357, 23200750): 1490932239, (198796, 23200751): 1490932236, (3942, 23200754): 34616858, (6844578, 23200753): 1490932231}, 'flows':{(11, 23200752): 1490932234, (3, 23200751): 1490932241, (14, 23200753): 1490932240, (26, 23200750): 1490932243, (166, 23200754): 1490932233}}, 'sipv4': {'pckts': {(784, 23200753): 1490932229, (4542, 23200755): 389939994, (6, 23200754): 1844882381, (7, 23200751): 3245197133, (-5, 23200750): 2130706433}, 'bytes': {(3709, 23200753): 1490932241, (878936, 23200755): 1156064098, (3942, 23200754): 1490932236, (1399, 23200752): 520838764, (6811172, 23200751): 389939994}, 'flows': {(6, 23200755): 3163192613, (3, 23200751): 1844846946, (25, 23200752): 1390085144, (11, 23200753): 1490932239, (67, 23200750): 2990491714}}}, 23200751: {'dipv4': {'pckts': {(3667, 23200754): 625850440, (25, 23200752): 1390091821, (-2, 23200751): 2130706433, (2, 23200756): 3639549972, (10, 23200753): 1390085476}, 'bytes': {(40272, 23200753): 1390085485, (159626, 23200752): 625850440, (117, 23200756): 3639549972, (-2, 23200751): 2130706433, (728, 23200754): 1390087394}, 'flows': {(3, 23200756): 1390087985, (1, 23200752): 831557494, (3, 23200753): 1490932233, (-2, 23200751): 2130706433, (2, 23200754): 2915181193}}, 'sipv4': {'pckts': {(25, 23200752): 3651343655, (-2, 23200751): 2130706433, (3667, 23200754): 1390127411, (10, 23200753): 1796712159, (93, 23200755): 2915181215}, 'bytes': {(19613, 23200753): 390746172, (117, 23200756): 1390134674, (40272, 23200752): 1122534314, (-2, 23200751): 2130706433, (159626, 23200754): 1390127411}, 'flows': {(2, 23200755): 134743044, (1, 23200752): 1390132074, (-2, 23200751): 2130706433, (1, 23200753): 1390085145, (2, 23200754): 3475948033}}}} 335 | two={0: {'dipv4': {'pckts': {(-5, 0): 2130706433, (-2, 0): 2130706433, (-1, 0): 2130706433, (-3, 0): 2130706433, (-4, 0): 2130706433}, 'bytes': {(-5, 0): 2130706433, (-2, 0): 2130706433, (-1, 0): 2130706433, (-3, 0): 2130706433, (-4, 0): 2130706433}, 'flows': {(-5, 0): 2130706433, (-2, 0): 2130706433, (-1, 0): 2130706433, (-3, 0): 2130706433, (-4, 0): 2130706433}}, 'sipv4': {'pckts': {(-5, 0): 2130706433, (-2, 0): 2130706433, (-1, 0): 2130706433, (-3, 0): 2130706433, (-4, 0): 2130706433}, 'bytes': {(-5, 0): 2130706433, (-2, 0): 2130706433, (-1, 0): 2130706433, (-3, 0): 2130706433, (-4, 0): 2130706433}, 'flows': {(-5, 0): 2130706433, (-2, 0): 2130706433, (-1, 0): 2130706433, (-3, 0): 2130706433, (-4, 0): 2130706433}}}, 1: {'dipv4': {'pckts': {(-3, 1): 2130706433, (-1, 1): 2130706433, (-5, 1): 2130706433, (-4, 1): 2130706433, (-2, 1): 2130706433}, 'bytes': {(-3, 1): 2130706433, (-1, 1): 2130706433, (-5, 1): 2130706433, (-4, 1): 2130706433, (-2, 1): 2130706433}, 'flows': {(-3, 1): 2130706433, (-1, 1): 2130706433, (-5, 1): 2130706433, (-4, 1): 2130706433, (-2, 1): 2130706433}}, 'sipv4': {'pckts': {(-3, 1): 2130706433, (-1, 1): 2130706433, (-5, 1): 2130706433, (-4, 1): 2130706433, (-2, 1): 2130706433}, 'bytes': {(-3, 1): 2130706433, (-1, 1): 2130706433, (-5, 1): 2130706433, (-4, 1): 2130706433, (-2, 1): 2130706433}, 'flows': {(-3, 1): 2130706433, (-1, 1): 2130706433, (-5, 1): 2130706433, (-4, 1): 2130706433, (-2, 1): 2130706433}}}, 2: {'dipv4': {'pckts': {(-4, 2): 2130706433, (-5, 2): 2130706433, (-1, 2): 2130706433, (-3, 2): 2130706433, (-2, 2): 2130706433}, 'bytes': {(-4, 2): 2130706433, (-5, 2): 2130706433, (-1, 2): 2130706433, (-3, 2): 2130706433, (-2, 2): 2130706433}, 'flows': {(-4, 2): 2130706433, (-5, 2): 2130706433, (-1, 2): 2130706433, (-3, 2): 2130706433, (-2, 2): 2130706433}}, 'sipv4': {'pckts': {(-4, 2): 2130706433, (-5, 2): 2130706433, (-1, 2): 2130706433, (-3, 2): 2130706433, (-2, 2): 2130706433}, 'bytes': {(-4, 2): 2130706433, (-5, 2): 2130706433, (-1, 2): 2130706433, (-3, 2): 2130706433, (-2, 2): 2130706433}, 'flows': {(-4, 2): 2130706433, (-5, 2): 2130706433, (-1, 2): 2130706433, (-3, 2): 2130706433, (-2, 2): 2130706433}}}, 3: {'dipv4': {'pckts': {(-1, 3): 2130706433, (-2, 3): 2130706433, (-5, 3): 2130706433, (-4, 3): 2130706433, (-3, 3): 2130706433}, 'bytes': {(-1, 3): 2130706433, (-2, 3): 2130706433, (-5, 3): 2130706433, (-4, 3): 2130706433, (-3, 3): 2130706433}, 'flows': {(-1, 3): 2130706433, (-2, 3): 2130706433, (-5, 3): 2130706433, (-4, 3): 2130706433, (-3, 3): 2130706433}}, 'sipv4': {'pckts': {(-1, 3): 2130706433, (-2, 3): 2130706433, (-5, 3): 2130706433, (-4, 3): 2130706433, (-3, 3): 2130706433}, 'bytes': {(-1, 3): 2130706433, (-2, 3): 2130706433, (-5, 3): 2130706433, (-4, 3): 2130706433, (-3, 3): 2130706433}, 'flows': {(-1, 3): 2130706433, (-2, 3): 2130706433, (-5, 3): 2130706433, (-4, 3): 2130706433, (-3, 3): 2130706433}}}, 4: {'dipv4': {'pckts': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'bytes': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'flows': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}}, 'sipv4': {'pckts': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'bytes': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}, 'flows': {(-1, 4): 2130706433, (-3, 4): 2130706433, (-4, 4): 2130706433, (-5, 4): 2130706433, (-2, 4): 2130706433}}}} 336 | for data in (one,two): 337 | print flow_traffic(data) 338 | -------------------------------------------------------------------------------- /lib/exaddos/ipfix.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | ipfix.py 4 | 5 | Created by Thomas Mangin on 2014-02-09. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import sys 10 | import struct 11 | import random 12 | 13 | # http://www.iana.org/assignments/ipfix/ipfix.xhtml 14 | 15 | class TEMPLATE (object): 16 | octetDeltaCount = 1 17 | packetDeltaCount = 2 18 | deltaFlowCount = 3 19 | protocolIdentifier = 4 20 | ipClassOfService = 5 21 | tcpControlBits = 6 22 | sourceTransportPort = 7 23 | sourceIPv4Address = 8 24 | sourceIPv4PrefixLength = 9 25 | ingressInterface = 10 26 | destinationTransportPort = 11 27 | destinationIPv4Address = 12 28 | destinationIPv4PrefixLength = 13 29 | egressInterface = 14 30 | ipNextHopIPv4Address = 15 31 | bgpSourceAsNumber = 16 32 | bgpDestinationAsNumber = 17 33 | bgpNextHopIPv4Address = 18 34 | # .... 35 | ipv6flowlabel = 31 36 | 37 | 38 | class PROTOCOL: 39 | ICMP = 1 40 | TCP = 6 41 | UDP = 17 42 | 43 | 44 | CONVERT = { 45 | (TEMPLATE.octetDeltaCount,1) : '>B', 46 | (TEMPLATE.octetDeltaCount,2) : '>H', 47 | (TEMPLATE.octetDeltaCount,4) : '>I', 48 | (TEMPLATE.octetDeltaCount,8) : '>Q', 49 | 50 | (TEMPLATE.packetDeltaCount,1) : '>B', 51 | (TEMPLATE.packetDeltaCount,2) : '>H', 52 | (TEMPLATE.packetDeltaCount,4) : '>I', 53 | (TEMPLATE.packetDeltaCount,8) : '>Q', 54 | 55 | (TEMPLATE.protocolIdentifier,1) : '>B', 56 | (TEMPLATE.protocolIdentifier,2) : '>H', 57 | (TEMPLATE.protocolIdentifier,4) : '>I', 58 | (TEMPLATE.protocolIdentifier,8) : '>Q', 59 | 60 | (TEMPLATE.sourceIPv4Address,1) : '>B', 61 | (TEMPLATE.sourceIPv4Address,2) : '>H', 62 | (TEMPLATE.sourceIPv4Address,4) : '>I', 63 | (TEMPLATE.sourceIPv4Address,8) : '>Q', 64 | 65 | (TEMPLATE.destinationIPv4Address,1) : '>B', 66 | (TEMPLATE.destinationIPv4Address,2) : '>H', 67 | (TEMPLATE.destinationIPv4Address,4) : '>I', 68 | (TEMPLATE.destinationIPv4Address,8) : '>Q', 69 | 70 | (TEMPLATE.sourceTransportPort,1) : '>B', 71 | (TEMPLATE.sourceTransportPort,2) : '>H', 72 | (TEMPLATE.sourceTransportPort,4) : '>I', 73 | (TEMPLATE.sourceTransportPort,8) : '>Q', 74 | 75 | (TEMPLATE.destinationTransportPort,1) : '>B', 76 | (TEMPLATE.destinationTransportPort,2) : '>H', 77 | (TEMPLATE.destinationTransportPort,4) : '>I', 78 | (TEMPLATE.destinationTransportPort,8) : '>Q', 79 | 80 | } 81 | 82 | NAME = { 83 | TEMPLATE.protocolIdentifier : 'proto', 84 | TEMPLATE.sourceIPv4Address : 'sipv4', 85 | TEMPLATE.destinationIPv4Address : 'dipv4', 86 | TEMPLATE.octetDeltaCount : 'bytes', 87 | TEMPLATE.packetDeltaCount : 'pckts', 88 | TEMPLATE.sourceTransportPort : 'sport', 89 | TEMPLATE.destinationTransportPort : 'dport', 90 | } 91 | 92 | class IPFIX (object): 93 | decoded = 0L 94 | 95 | care = [ 96 | TEMPLATE.protocolIdentifier, 97 | TEMPLATE.sourceIPv4Address, 98 | TEMPLATE.destinationIPv4Address, 99 | TEMPLATE.octetDeltaCount, 100 | TEMPLATE.packetDeltaCount, 101 | TEMPLATE.sourceTransportPort, 102 | TEMPLATE.destinationTransportPort, 103 | ] 104 | 105 | def __init__ (self,callback): 106 | self.template = {} 107 | self.callback = callback 108 | self.id = random.randint(1,999) 109 | 110 | # The format of the IPFIX Message Header 111 | # 0 1 2 3 112 | # 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 113 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 114 | # | Version Number | Length | 115 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 116 | # | Export Time | 117 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 118 | # | Sequence Number | 119 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 120 | # | Observation Domain ID | 121 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 122 | 123 | def read (self,data): 124 | version, length, epoch, sequence, domain = struct.unpack(">HHIII", data[:16]) 125 | 126 | if version != 10: 127 | return 128 | 129 | if length > len(data): 130 | return 131 | 132 | return self.read_set(epoch,data[16:]) 133 | 134 | # SET Header 135 | # 0 1 2 3 136 | # 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 137 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 138 | # | Set ID | Length | 139 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 140 | 141 | def read_set (self,epoch,data): 142 | if not data: 143 | return 144 | 145 | ID, length = struct.unpack(">HH", data[:4]) 146 | 147 | if ID >= 256: 148 | return self.read_data(ID,epoch,data[4:]) 149 | if ID == 2: 150 | return self.read_template(epoch,data[4:]) 151 | if ID == 3: 152 | return self.read_option_template(epoch,data[4:]) 153 | 154 | # ID 0 .. 1 : not used historical reasons 155 | # ID 4 .. 255 : reserved 156 | 157 | # Template Record Header 158 | 159 | # 0 1 2 3 160 | # 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 161 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 162 | # | Template ID (> 255) | Field Count | 163 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 164 | 165 | # The Field Specifier format is shown in Figure G. 166 | 167 | # 0 1 2 3 168 | # 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 169 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 170 | # |E| Information Element ident. | Field Length | 171 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 172 | # | Enterprise Number | 173 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 174 | 175 | def read_template (self,epoch,data): 176 | template, number = struct.unpack(">HH", data[:4]) 177 | 178 | # we do not give a damn about enterprise data 179 | if template & 0b1000000000000000: # 1 << 15 180 | #enterprise = struct.unpack(">I",data[4:8]) 181 | return self.read_set(epoch,data[8:]) 182 | 183 | # deleting known template 184 | if number == 0: 185 | if template in self.template: 186 | del self.template[template] 187 | return self.read_set(epoch,data[4:]) 188 | 189 | format = {} 190 | offset = 0 191 | 192 | for index in range(number): 193 | what, size = struct.unpack(">HH", data[4+(index*4):4+(index*4)+4]) 194 | if what in self.care: 195 | format[what] = (offset,size) 196 | offset += size 197 | 198 | # the template has what we care about :-) 199 | if len(format) == len(self.care): 200 | self.template[template] = format 201 | 202 | return self.read_set(epoch,data[4 + (number*4):]) 203 | 204 | # 0 1 2 3 205 | # 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 206 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 207 | # | Template ID (> 255) | Field Count | 208 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 209 | # | Scope Field Count | 210 | # +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 211 | 212 | # +--------------------------------------------------+ 213 | # | Options Template Record Header | 214 | # +--------------------------------------------------+ 215 | # | Field Specifier | 216 | # +--------------------------------------------------+ 217 | # | Field Specifier | 218 | # +--------------------------------------------------+ 219 | # ... 220 | # +--------------------------------------------------+ 221 | # | Field Specifier | 222 | # +--------------------------------------------------+ 223 | 224 | def read_option_template (self,epoch,data): 225 | template, number, scope = struct.unpack(">HHH", data[:6]) 226 | 227 | # as we do not really care :-) 228 | # Juniper return number*4 + scope*2 ! 229 | return self.read_set(epoch,data[6+(number*4)+(scope*2):]) 230 | 231 | # # deleting known template 232 | # if number == 0: 233 | # if template in self.template: 234 | # del self.template[template] 235 | # return self.read_set(epoch,data[4:]) 236 | 237 | # # invalid data 238 | # if scope == 0 or scope > number: 239 | # return 240 | 241 | # # process scope fields 242 | # for _ in range (scope): 243 | # what, size = struct.unpack(">HH", data[:4]) 244 | # pass 245 | 246 | # # process "normal" options fields 247 | # for _ in range (number-scope): 248 | # what, size = struct.unpack(">HH", data[:4]) 249 | # pass 250 | 251 | # #return self.read_set(epoch,data[6+(number*4):]) 252 | 253 | def read_data (self,setid,epoch,data): 254 | # unknown template, ignore it ! 255 | if setid not in self.template: 256 | return 257 | 258 | extracted = {'epoch':epoch,'flows':1} 259 | format = self.template[setid] 260 | 261 | for what in format: 262 | offset,size = format[what] 263 | extracted[NAME[what]], = struct.unpack(CONVERT[(what,size)],data[offset:offset+size]) 264 | 265 | # # reports the data decoding rate per thread 266 | # self.decoded +=1 267 | # if not self.decoded % 1000: 268 | # print "id %d decoded %ld flows" % (self.id,self.decoded) 269 | # sys.stdout.flush() 270 | # if self.decoded == sys.maxint: 271 | # self.decoded = self.decoded % 1000 272 | 273 | self.callback(extracted) 274 | -------------------------------------------------------------------------------- /lib/exaddos/leak/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Exa-Networks/exaddos/15c5e0a5575bab4c8fbac31eeb6790eb9b710a39/lib/exaddos/leak/__init__.py -------------------------------------------------------------------------------- /lib/exaddos/leak/gcdump.py: -------------------------------------------------------------------------------- 1 | # http://teethgrinder.co.uk/perm.php?a=Python-memory-leak-detector 2 | import gc 3 | import inspect 4 | 5 | def dump(): 6 | # force collection 7 | print "\nCollecting GARBAGE:" 8 | gc.collect() 9 | # prove they have been collected 10 | print "\nCollecting GARBAGE:" 11 | gc.collect() 12 | 13 | print "\nGARBAGE OBJECTS:" 14 | for x in gc.garbage: 15 | s = str(x) 16 | if len(s) > 80: s = "%s..." % s[:80] 17 | 18 | print "::", s 19 | print " type:", type(x) 20 | print " referrers:", len(gc.get_referrers(x)) 21 | try: 22 | print " is class:", inspect.isclass(type(x)) 23 | print " module:", inspect.getmodule(x) 24 | 25 | lines, line_num = inspect.getsourcelines(type(x)) 26 | print " line num:", line_num 27 | for l in lines: 28 | print " line:", l.rstrip("\n") 29 | except: 30 | pass 31 | 32 | print 33 | 34 | class tmp(object): 35 | def __init__(self): 36 | a = 0 37 | 38 | if __name__=="__main__": 39 | import gc 40 | gc.enable() 41 | gc.set_debug(gc.DEBUG_LEAK) 42 | 43 | # make a leak 44 | l = [tmp()] 45 | l.append(l) 46 | del l 47 | 48 | dump_garbage() 49 | 50 | 51 | -------------------------------------------------------------------------------- /lib/exaddos/leak/objgraph.py: -------------------------------------------------------------------------------- 1 | """ 2 | Tools for drawing Python object reference graphs with graphviz. 3 | 4 | You can find documentation online at http://mg.pov.lt/objgraph/ 5 | 6 | Copyright (c) 2008-2010 Marius Gedminas 7 | Copyright (c) 2010 Stefano Rivera 8 | 9 | Released under the MIT licence. 10 | """ 11 | # Permission is hereby granted, free of charge, to any person obtaining a 12 | # copy of this software and associated documentation files (the "Software"), 13 | # to deal in the Software without restriction, including without limitation 14 | # the rights to use, copy, modify, merge, publish, distribute, sublicense, 15 | # and/or sell copies of the Software, and to permit persons to whom the 16 | # Software is furnished to do so, subject to the following conditions: 17 | # 18 | # The above copyright notice and this permission notice shall be included in 19 | # all copies or substantial portions of the Software. 20 | # 21 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 22 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 23 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 24 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 25 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 26 | # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 27 | # DEALINGS IN THE SOFTWARE. 28 | 29 | __author__ = "Marius Gedminas (marius@gedmin.as)" 30 | __copyright__ = "Copyright (c) 2008-2011 Marius Gedminas" 31 | __license__ = "MIT" 32 | __version__ = "1.7.1" 33 | __date__ = "2011-12-11" 34 | 35 | 36 | import codecs 37 | import gc 38 | import re 39 | import inspect 40 | import types 41 | import operator 42 | import os 43 | import subprocess 44 | import tempfile 45 | import sys 46 | import itertools 47 | 48 | 49 | try: 50 | basestring 51 | except NameError: 52 | # Python 3.x compatibility 53 | basestring = str 54 | 55 | try: 56 | iteritems = dict.iteritems 57 | except AttributeError: 58 | # Python 3.x compatibility 59 | iteritems = dict.items 60 | 61 | 62 | def count(typename, objects=None): 63 | """Count objects tracked by the garbage collector with a given class name. 64 | 65 | Example: 66 | 67 | >>> count('dict') 68 | 42 69 | >>> count('MyClass', get_leaking_objects()) 70 | 3 71 | 72 | Note that the GC does not track simple objects like int or str. 73 | 74 | .. versionchanged:: 1.7 75 | New parameter: ``objects``. 76 | 77 | """ 78 | if objects is None: 79 | objects = gc.get_objects() 80 | return sum(1 for o in objects if type(o).__name__ == typename) 81 | 82 | 83 | def typestats(objects=None): 84 | """Count the number of instances for each type tracked by the GC. 85 | 86 | Note that the GC does not track simple objects like int or str. 87 | 88 | Note that classes with the same name but defined in different modules 89 | will be lumped together. 90 | 91 | Example: 92 | 93 | >>> typestats() 94 | {'list': 12041, 'tuple': 10245, ...} 95 | >>> typestats(get_leaking_objects()) 96 | {'MemoryError': 1, 'tuple': 2795, 'RuntimeError': 1, 'list': 47, ...} 97 | 98 | .. versionadded:: 1.1 99 | 100 | .. versionchanged:: 1.7 101 | New parameter: ``objects``. 102 | 103 | """ 104 | if objects is None: 105 | objects = gc.get_objects() 106 | stats = {} 107 | for o in objects: 108 | stats.setdefault(type(o).__name__, 0) 109 | stats[type(o).__name__] += 1 110 | return stats 111 | 112 | 113 | def most_common_types(limit=10, objects=None): 114 | """Count the names of types with the most instances. 115 | 116 | Returns a list of (type_name, count), sorted most-frequent-first. 117 | 118 | Limits the return value to at most ``limit`` items. You may set ``limit`` 119 | to None to avoid that. 120 | 121 | The caveats documented in :func:`typestats` apply. 122 | 123 | Example: 124 | 125 | >>> most_common_types(limit=2) 126 | [('list', 12041), ('tuple', 10245)] 127 | 128 | .. versionadded:: 1.4 129 | 130 | .. versionchanged:: 1.7 131 | New parameter: ``objects``. 132 | 133 | """ 134 | stats = sorted(typestats(objects).items(), key=operator.itemgetter(1), 135 | reverse=True) 136 | if limit: 137 | stats = stats[:limit] 138 | return stats 139 | 140 | 141 | def show_most_common_types(limit=10, objects=None): 142 | """Print the table of types of most common instances. 143 | 144 | The caveats documented in :func:`typestats` apply. 145 | 146 | Example: 147 | 148 | >>> show_most_common_types(limit=5) 149 | tuple 8959 150 | function 2442 151 | wrapper_descriptor 1048 152 | dict 953 153 | builtin_function_or_method 800 154 | 155 | .. versionadded:: 1.1 156 | 157 | .. versionchanged:: 1.7 158 | New parameter: ``objects``. 159 | 160 | """ 161 | stats = most_common_types(limit, objects) 162 | width = max(len(name) for name, count in stats) 163 | for name, count in stats: 164 | print('%-*s %i' % (width, name, count)) 165 | 166 | 167 | def show_growth(limit=10, peak_stats={}): 168 | """Show the increase in peak object counts since last call. 169 | 170 | Limits the output to ``limit`` largest deltas. You may set ``limit`` to 171 | None to see all of them. 172 | 173 | Uses and updates ``peak_stats``, a dictionary from type names to previously 174 | seen peak object counts. Usually you don't need to pay attention to this 175 | argument. 176 | 177 | The caveats documented in :func:`typestats` apply. 178 | 179 | Example: 180 | 181 | >>> objgraph.show_growth() 182 | wrapper_descriptor 970 +14 183 | tuple 12282 +10 184 | dict 1922 +7 185 | ... 186 | 187 | .. versionadded:: 1.5 188 | """ 189 | gc.collect() 190 | stats = typestats() 191 | deltas = {} 192 | for name, count in iteritems(stats): 193 | old_count = peak_stats.get(name, 0) 194 | if count > old_count: 195 | deltas[name] = count - old_count 196 | peak_stats[name] = count 197 | deltas = sorted(deltas.items(), key=operator.itemgetter(1), 198 | reverse=True) 199 | if limit: 200 | deltas = deltas[:limit] 201 | if deltas: 202 | width = max(len(name) for name, count in deltas) 203 | for name, delta in deltas: 204 | print('%-*s%9d %+9d' % (width, name, stats[name], delta)) 205 | 206 | 207 | def get_leaking_objects(objects=None): 208 | """Return objects that do not have any referents. 209 | 210 | These could indicate reference-counting bugs in C code. Or they could 211 | be legitimate. 212 | 213 | Note that the GC does not track simple objects like int or str. 214 | 215 | .. versionadded:: 1.7 216 | """ 217 | if objects is None: 218 | gc.collect() 219 | objects = gc.get_objects() 220 | try: 221 | ids = set(id(i) for i in objects) 222 | for i in objects: 223 | ids.difference_update(id(j) for j in gc.get_referents(i)) 224 | # this then is our set of objects without referrers 225 | return [i for i in objects if id(i) in ids] 226 | finally: 227 | objects = i = j = None # clear cyclic references to frame 228 | 229 | 230 | def by_type(typename, objects=None): 231 | """Return objects tracked by the garbage collector with a given class name. 232 | 233 | Example: 234 | 235 | >>> by_type('MyClass') 236 | [] 237 | 238 | Note that the GC does not track simple objects like int or str. 239 | 240 | .. versionchanged:: 1.7 241 | New parameter: ``objects``. 242 | 243 | """ 244 | if objects is None: 245 | objects = gc.get_objects() 246 | return [o for o in objects if type(o).__name__ == typename] 247 | 248 | 249 | def at(addr): 250 | """Return an object at a given memory address. 251 | 252 | The reverse of id(obj): 253 | 254 | >>> at(id(obj)) is obj 255 | True 256 | 257 | Note that this function does not work on objects that are not tracked by 258 | the GC (e.g. ints or strings). 259 | """ 260 | for o in gc.get_objects(): 261 | if id(o) == addr: 262 | return o 263 | return None 264 | 265 | 266 | def find_ref_chain(obj, predicate, max_depth=20, extra_ignore=()): 267 | """Find a shortest chain of references leading from obj. 268 | 269 | The end of the chain will be some object that matches your predicate. 270 | 271 | ``predicate`` is a function taking one argument and returning a boolean. 272 | 273 | ``max_depth`` limits the search depth. 274 | 275 | ``extra_ignore`` can be a list of object IDs to exclude those objects from 276 | your search. 277 | 278 | Example: 279 | 280 | >>> find_chain(obj, lambda x: isinstance(x, MyClass)) 281 | [obj, ..., ] 282 | 283 | Returns ``[obj]`` if such a chain could not be found. 284 | 285 | .. versionadded:: 1.7 286 | """ 287 | return find_chain(obj, predicate, gc.get_referents, 288 | max_depth=max_depth, extra_ignore=extra_ignore)[::-1] 289 | 290 | 291 | def find_backref_chain(obj, predicate, max_depth=20, extra_ignore=()): 292 | """Find a shortest chain of references leading to obj. 293 | 294 | The start of the chain will be some object that matches your predicate. 295 | 296 | ``predicate`` is a function taking one argument and returning a boolean. 297 | 298 | ``max_depth`` limits the search depth. 299 | 300 | ``extra_ignore`` can be a list of object IDs to exclude those objects from 301 | your search. 302 | 303 | Example: 304 | 305 | >>> find_backref_chain(obj, inspect.ismodule) 306 | [, ..., obj] 307 | 308 | Returns ``[obj]`` if such a chain could not be found. 309 | 310 | .. versionchanged:: 1.5 311 | Returns ``obj`` instead of ``None`` when a chain could not be found. 312 | 313 | """ 314 | return find_chain(obj, predicate, gc.get_referrers, 315 | max_depth=max_depth, extra_ignore=extra_ignore) 316 | 317 | 318 | def show_backrefs(objs, max_depth=3, extra_ignore=(), filter=None, too_many=10, 319 | highlight=None, filename=None, extra_info=None, 320 | refcounts=False): 321 | """Generate an object reference graph ending at ``objs``. 322 | 323 | The graph will show you what objects refer to ``objs``, directly and 324 | indirectly. 325 | 326 | ``objs`` can be a single object, or it can be a list of objects. If 327 | unsure, wrap the single object in a new list. 328 | 329 | ``filename`` if specified, can be the name of a .dot or a .png file, 330 | indicating the desired output format. If not specified, ``show_backrefs`` 331 | will try to produce a .dot file and spawn a viewer (xdot). If xdot is 332 | not available, ``show_backrefs`` will convert the .dot file to a .png 333 | and print its name. 334 | 335 | Use ``max_depth`` and ``too_many`` to limit the depth and breadth of the 336 | graph. 337 | 338 | Use ``filter`` (a predicate) and ``extra_ignore`` (a list of object IDs) to 339 | remove undesired objects from the graph. 340 | 341 | Use ``highlight`` (a predicate) to highlight certain graph nodes in blue. 342 | 343 | Use ``extra_info`` (a function taking one argument and returning a 344 | string) to report extra information for objects. 345 | 346 | Specify ``refcounts=True`` if you want to see reference counts. 347 | These will mostly match the number of arrows pointing to an object, 348 | but can be different for various reasons. 349 | 350 | Examples: 351 | 352 | >>> show_backrefs(obj) 353 | >>> show_backrefs([obj1, obj2]) 354 | >>> show_backrefs(obj, max_depth=5) 355 | >>> show_backrefs(obj, filter=lambda x: not inspect.isclass(x)) 356 | >>> show_backrefs(obj, highlight=inspect.isclass) 357 | >>> show_backrefs(obj, extra_ignore=[id(locals())]) 358 | 359 | .. versionchanged:: 1.3 360 | New parameters: ``filename``, ``extra_info``. 361 | 362 | .. versionchanged:: 1.5 363 | New parameter: ``refcounts``. 364 | 365 | """ 366 | show_graph(objs, max_depth=max_depth, extra_ignore=extra_ignore, 367 | filter=filter, too_many=too_many, highlight=highlight, 368 | edge_func=gc.get_referrers, swap_source_target=False, 369 | filename=filename, extra_info=extra_info, refcounts=refcounts) 370 | 371 | 372 | def show_refs(objs, max_depth=3, extra_ignore=(), filter=None, too_many=10, 373 | highlight=None, filename=None, extra_info=None, 374 | refcounts=False): 375 | """Generate an object reference graph starting at ``objs``. 376 | 377 | The graph will show you what objects are reachable from ``objs``, directly 378 | and indirectly. 379 | 380 | ``objs`` can be a single object, or it can be a list of objects. If 381 | unsure, wrap the single object in a new list. 382 | 383 | ``filename`` if specified, can be the name of a .dot or a .png file, 384 | indicating the desired output format. If not specified, ``show_refs`` 385 | will try to produce a .dot file and spawn a viewer (xdot). If xdot is 386 | not available, ``show_refs`` will convert the .dot file to a .png 387 | and print its name. 388 | 389 | Use ``max_depth`` and ``too_many`` to limit the depth and breadth of the 390 | graph. 391 | 392 | Use ``filter`` (a predicate) and ``extra_ignore`` (a list of object IDs) to 393 | remove undesired objects from the graph. 394 | 395 | Use ``highlight`` (a predicate) to highlight certain graph nodes in blue. 396 | 397 | Use ``extra_info`` (a function returning a string) to report extra 398 | information for objects. 399 | 400 | Specify ``refcounts=True`` if you want to see reference counts. 401 | 402 | Examples: 403 | 404 | >>> show_refs(obj) 405 | >>> show_refs([obj1, obj2]) 406 | >>> show_refs(obj, max_depth=5) 407 | >>> show_refs(obj, filter=lambda x: not inspect.isclass(x)) 408 | >>> show_refs(obj, highlight=inspect.isclass) 409 | >>> show_refs(obj, extra_ignore=[id(locals())]) 410 | 411 | .. versionadded:: 1.1 412 | 413 | .. versionchanged:: 1.3 414 | New parameters: ``filename``, ``extra_info``. 415 | 416 | .. versionchanged:: 1.5 417 | New parameter: ``refcounts``. 418 | Follows references from module objects instead of stopping. 419 | 420 | """ 421 | show_graph(objs, max_depth=max_depth, extra_ignore=extra_ignore, 422 | filter=filter, too_many=too_many, highlight=highlight, 423 | edge_func=gc.get_referents, swap_source_target=True, 424 | filename=filename, extra_info=extra_info, refcounts=refcounts) 425 | 426 | 427 | def show_chain(*chains, **kw): 428 | """Show a chain (or several chains) of object references. 429 | 430 | Useful in combination with :func:`find_ref_chain` or 431 | :func:`find_backref_chain`, e.g. 432 | 433 | >>> show_chain(find_backref_chain(obj, inspect.ismodule)) 434 | 435 | You can specify if you want that chain traced backwards or forwards 436 | by passing a ``backrefs`` keyword argument, e.g. 437 | 438 | >>> show_chain(find_ref_chain(obj, inspect.ismodule), 439 | ... backrefs=False) 440 | 441 | Ideally this shouldn't matter, but for some objects 442 | :func:`gc.get_referrers` and :func:`gc.get_referents` are not perfectly 443 | symmetrical. 444 | 445 | You can specify ``highlight``, ``extra_info`` or ``filename`` arguments 446 | like for :func:`show_backrefs` or :func:`show_refs`. 447 | 448 | .. versionadded:: 1.5 449 | 450 | .. versionchanged:: 1.7 451 | New parameter: ``backrefs``. 452 | 453 | """ 454 | backrefs = kw.pop('backrefs', True) 455 | chains = [chain for chain in chains if chain] # remove empty ones 456 | def in_chains(x, ids=set(map(id, itertools.chain(*chains)))): 457 | return id(x) in ids 458 | max_depth = max(map(len, chains)) - 1 459 | if backrefs: 460 | show_backrefs([chain[-1] for chain in chains], max_depth=max_depth, 461 | filter=in_chains, **kw) 462 | else: 463 | show_refs([chain[0] for chain in chains], max_depth=max_depth, 464 | filter=in_chains, **kw) 465 | 466 | # 467 | # Internal helpers 468 | # 469 | 470 | def find_chain(obj, predicate, edge_func, max_depth=20, extra_ignore=()): 471 | queue = [obj] 472 | depth = {id(obj): 0} 473 | parent = {id(obj): None} 474 | ignore = set(extra_ignore) 475 | ignore.add(id(extra_ignore)) 476 | ignore.add(id(queue)) 477 | ignore.add(id(depth)) 478 | ignore.add(id(parent)) 479 | ignore.add(id(ignore)) 480 | ignore.add(id(sys._getframe())) # this function 481 | ignore.add(id(sys._getframe(1))) # find_chain/find_backref_chain, most likely 482 | gc.collect() 483 | while queue: 484 | target = queue.pop(0) 485 | if predicate(target): 486 | chain = [target] 487 | while parent[id(target)] is not None: 488 | target = parent[id(target)] 489 | chain.append(target) 490 | return chain 491 | tdepth = depth[id(target)] 492 | if tdepth < max_depth: 493 | referrers = edge_func(target) 494 | ignore.add(id(referrers)) 495 | for source in referrers: 496 | if id(source) in ignore: 497 | continue 498 | if id(source) not in depth: 499 | depth[id(source)] = tdepth + 1 500 | parent[id(source)] = target 501 | queue.append(source) 502 | return [obj] # not found 503 | 504 | 505 | def show_graph(objs, edge_func, swap_source_target, 506 | max_depth=3, extra_ignore=(), filter=None, too_many=10, 507 | highlight=None, filename=None, extra_info=None, 508 | refcounts=False): 509 | if not isinstance(objs, (list, tuple)): 510 | objs = [objs] 511 | if filename and filename.endswith('.dot'): 512 | f = codecs.open(filename, 'w', encoding='utf-8') 513 | dot_filename = filename 514 | else: 515 | fd, dot_filename = tempfile.mkstemp('.dot', text=True) 516 | f = os.fdopen(fd, "w") 517 | if f.encoding != None: 518 | # Python 3 will wrap the file in the user's preferred encoding 519 | # Re-wrap it for utf-8 520 | import io 521 | f = io.TextIOWrapper(f.detach(), 'utf-8') 522 | f.write('digraph ObjectGraph {\n' 523 | ' node[shape=box, style=filled, fillcolor=white];\n') 524 | queue = [] 525 | depth = {} 526 | ignore = set(extra_ignore) 527 | ignore.add(id(objs)) 528 | ignore.add(id(extra_ignore)) 529 | ignore.add(id(queue)) 530 | ignore.add(id(depth)) 531 | ignore.add(id(ignore)) 532 | ignore.add(id(sys._getframe())) # this function 533 | ignore.add(id(sys._getframe(1))) # show_refs/show_backrefs, most likely 534 | for obj in objs: 535 | f.write(' %s[fontcolor=red];\n' % (obj_node_id(obj))) 536 | depth[id(obj)] = 0 537 | queue.append(obj) 538 | del obj 539 | gc.collect() 540 | nodes = 0 541 | while queue: 542 | nodes += 1 543 | target = queue.pop(0) 544 | tdepth = depth[id(target)] 545 | f.write(' %s[label="%s"];\n' % (obj_node_id(target), obj_label(target, extra_info, refcounts))) 546 | h, s, v = gradient((0, 0, 1), (0, 0, .3), tdepth, max_depth) 547 | if inspect.ismodule(target): 548 | h = .3 549 | s = 1 550 | if highlight and highlight(target): 551 | h = .6 552 | s = .6 553 | v = 0.5 + v * 0.5 554 | f.write(' %s[fillcolor="%g,%g,%g"];\n' % (obj_node_id(target), h, s, v)) 555 | if v < 0.5: 556 | f.write(' %s[fontcolor=white];\n' % (obj_node_id(target))) 557 | if hasattr(getattr(target, '__class__', None), '__del__'): 558 | f.write(" %s->%s_has_a_del[color=red,style=dotted,len=0.25,weight=10];\n" % (obj_node_id(target), obj_node_id(target))) 559 | f.write(' %s_has_a_del[label="__del__",shape=doublecircle,height=0.25,color=red,fillcolor="0,.5,1",fontsize=6];\n' % (obj_node_id(target))) 560 | if tdepth >= max_depth: 561 | continue 562 | if inspect.ismodule(target) and not swap_source_target: 563 | # For show_backrefs(), it makes sense to stop when reaching a 564 | # module because you'll end up in sys.modules and explode the 565 | # graph with useless clutter. For show_refs(), it makes sense 566 | # to continue. 567 | continue 568 | neighbours = edge_func(target) 569 | ignore.add(id(neighbours)) 570 | n = 0 571 | skipped = 0 572 | for source in neighbours: 573 | if id(source) in ignore: 574 | continue 575 | if filter and not filter(source): 576 | continue 577 | if n >= too_many: 578 | skipped += 1 579 | continue 580 | if swap_source_target: 581 | srcnode, tgtnode = target, source 582 | else: 583 | srcnode, tgtnode = source, target 584 | elabel = edge_label(srcnode, tgtnode) 585 | f.write(' %s -> %s%s;\n' % (obj_node_id(srcnode), obj_node_id(tgtnode), elabel)) 586 | if id(source) not in depth: 587 | depth[id(source)] = tdepth + 1 588 | queue.append(source) 589 | n += 1 590 | del source 591 | del neighbours 592 | if skipped > 0: 593 | h, s, v = gradient((0, 1, 1), (0, 1, .3), tdepth + 1, max_depth) 594 | if swap_source_target: 595 | label = "%d more references" % skipped 596 | edge = "%s->too_many_%s" % (obj_node_id(target), obj_node_id(target)) 597 | else: 598 | label = "%d more backreferences" % skipped 599 | edge = "too_many_%s->%s" % (obj_node_id(target), obj_node_id(target)) 600 | f.write(' %s[color=red,style=dotted,len=0.25,weight=10];\n' % edge) 601 | f.write(' too_many_%s[label="%s",shape=box,height=0.25,color=red,fillcolor="%g,%g,%g",fontsize=6];\n' % (obj_node_id(target), label, h, s, v)) 602 | f.write(' too_many_%s[fontcolor=white];\n' % (obj_node_id(target))) 603 | f.write("}\n") 604 | f.close() 605 | print("Graph written to %s (%d nodes)" % (dot_filename, nodes)) 606 | if filename and filename.endswith('.dot'): 607 | # nothing else to do, the user asked for a .dot file 608 | return 609 | if not filename and program_in_path('xdot'): 610 | print("Spawning graph viewer (xdot)") 611 | subprocess.Popen(['xdot', dot_filename], close_fds=True) 612 | elif program_in_path('dot'): 613 | if not filename: 614 | print("Graph viewer (xdot) not found, generating a png instead") 615 | if filename and filename.endswith('.png'): 616 | f = open(filename, 'wb') 617 | png_filename = filename 618 | else: 619 | if filename: 620 | print("Unrecognized file type (%s)" % filename) 621 | fd, png_filename = tempfile.mkstemp('.png', text=False) 622 | f = os.fdopen(fd, "wb") 623 | dot = subprocess.Popen(['dot', '-Tpng', dot_filename], 624 | stdout=f, close_fds=False) 625 | dot.wait() 626 | f.close() 627 | print("Image generated as %s" % png_filename) 628 | else: 629 | if filename: 630 | print("Graph viewer (xdot) and image renderer (dot) not found, not doing anything else") 631 | else: 632 | print("Unrecognized file type (%s), not doing anything else" % filename) 633 | 634 | 635 | def obj_node_id(obj): 636 | return ('o%d' % id(obj)).replace('-', '_') 637 | 638 | 639 | def obj_label(obj, extra_info=None, refcounts=False): 640 | label = [type(obj).__name__] 641 | if refcounts: 642 | label[0] += ' [%d]' % (sys.getrefcount(obj) - 4) 643 | # Why -4? To ignore the references coming from 644 | # obj_label's frame (obj) 645 | # show_graph's frame (target variable) 646 | # sys.getrefcount()'s argument 647 | # something else that doesn't show up in gc.get_referrers() 648 | label.append(safe_repr(obj)) 649 | if extra_info: 650 | label.append(str(extra_info(obj))) 651 | return quote('\n'.join(label)) 652 | 653 | 654 | def quote(s): 655 | return (s.replace("\\", "\\\\") 656 | .replace("\"", "\\\"") 657 | .replace("\n", "\\n") 658 | .replace("\0", "\\\\0")) 659 | 660 | 661 | def safe_repr(obj): 662 | try: 663 | return short_repr(obj) 664 | except: 665 | return '(unrepresentable)' 666 | 667 | 668 | def short_repr(obj): 669 | if isinstance(obj, (type, types.ModuleType, types.BuiltinMethodType, 670 | types.BuiltinFunctionType)): 671 | return obj.__name__ 672 | if isinstance(obj, types.MethodType): 673 | try: 674 | if obj.__self__ is not None: 675 | return obj.__func__.__name__ + ' (bound)' 676 | else: 677 | return obj.__func__.__name__ 678 | except AttributeError: 679 | # Python < 2.6 compatibility 680 | if obj.im_self is not None: 681 | return obj.im_func.__name__ + ' (bound)' 682 | else: 683 | return obj.im_func.__name__ 684 | 685 | if isinstance(obj, types.FrameType): 686 | return '%s:%s' % (obj.f_code.co_filename, obj.f_lineno) 687 | if isinstance(obj, (tuple, list, dict, set)): 688 | return '%d items' % len(obj) 689 | return repr(obj)[:40] 690 | 691 | 692 | def gradient(start_color, end_color, depth, max_depth): 693 | if max_depth == 0: 694 | # avoid division by zero 695 | return start_color 696 | h1, s1, v1 = start_color 697 | h2, s2, v2 = end_color 698 | f = float(depth) / max_depth 699 | h = h1 * (1-f) + h2 * f 700 | s = s1 * (1-f) + s2 * f 701 | v = v1 * (1-f) + v2 * f 702 | return h, s, v 703 | 704 | 705 | def edge_label(source, target): 706 | if isinstance(target, dict) and target is getattr(source, '__dict__', None): 707 | return ' [label="__dict__",weight=10]' 708 | if isinstance(source, types.FrameType): 709 | if target is source.f_locals: 710 | return ' [label="f_locals",weight=10]' 711 | if target is source.f_globals: 712 | return ' [label="f_globals",weight=10]' 713 | if isinstance(source, types.MethodType): 714 | try: 715 | if target is source.__self__: 716 | return ' [label="__self__",weight=10]' 717 | if target is source.__func__: 718 | return ' [label="__func__",weight=10]' 719 | except AttributeError: 720 | # Python < 2.6 compatibility 721 | if target is source.im_self: 722 | return ' [label="im_self",weight=10]' 723 | if target is source.im_func: 724 | return ' [label="im_func",weight=10]' 725 | if isinstance(source, types.FunctionType): 726 | for k in dir(source): 727 | if target is getattr(source, k): 728 | return ' [label="%s",weight=10]' % quote(k) 729 | if isinstance(source, dict): 730 | for k, v in iteritems(source): 731 | if v is target: 732 | if isinstance(k, basestring) and is_identifier(k): 733 | return ' [label="%s",weight=2]' % quote(k) 734 | else: 735 | return ' [label="%s"]' % quote(type(k).__name__ + "\n" 736 | + safe_repr(k)) 737 | return '' 738 | 739 | 740 | is_identifier = re.compile('[a-zA-Z_][a-zA-Z_0-9]*$').match 741 | 742 | 743 | def program_in_path(program): 744 | path = os.environ.get("PATH", os.defpath).split(os.pathsep) 745 | path = [os.path.join(dir, program) for dir in path] 746 | path = [True for file in path 747 | if os.path.isfile(file) or os.path.isfile(file + '.exe')] 748 | return bool(path) 749 | -------------------------------------------------------------------------------- /lib/exaddos/log.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | log.py 4 | 5 | Created by Thomas Mangin on 2014-02-18. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import sys 10 | from threading import Lock 11 | 12 | 13 | __loud = { 14 | 'stdout': True, 15 | 'stderr': True, 16 | } 17 | 18 | __lock = Lock() 19 | 20 | def log (string): 21 | if __loud['stdout']: 22 | with __lock: 23 | sys.stdout.write('%s\n' % string) 24 | sys.stdout.flush() 25 | 26 | def err (string): 27 | if __loud['stderr']: 28 | with __lock: 29 | sys.stderr.write('%s\n' % string) 30 | sys.stderr.flush() 31 | 32 | def silence (): 33 | __loud['stdout'] = False 34 | -------------------------------------------------------------------------------- /lib/exaddos/q.py: -------------------------------------------------------------------------------- 1 | from collections import deque 2 | import time 3 | 4 | class Empty (Exception): 5 | pass 6 | 7 | class Queue(): 8 | def __init__ (self): 9 | self.queue = deque() 10 | 11 | def qsize (self): 12 | return len(self.queue) 13 | 14 | def put (self, message): 15 | self.queue.append(message) 16 | 17 | def get (self, timeout=None): 18 | try: 19 | if self.queue: 20 | return self.queue.popleft() 21 | except IndexError: 22 | pass 23 | 24 | delay = 0.0005 25 | start = time.time() 26 | 27 | running = True 28 | 29 | while running: 30 | try: 31 | while True: 32 | if timeout: 33 | if time.time() > start + timeout: 34 | running = False 35 | break 36 | 37 | delay = min(0.05, 2*delay) 38 | time.sleep(delay) 39 | 40 | if self.queue: 41 | return self.queue.popleft() 42 | except IndexError: 43 | pass 44 | 45 | raise Empty 46 | 47 | 48 | if __name__ == '__main__': 49 | q = Queue() 50 | q.put('foo') 51 | q.put('bar') 52 | print q.get(1) 53 | print q.get(1) 54 | try: 55 | q.put('yay') 56 | print q.get(1) 57 | print q.get(2) 58 | except: 59 | print 'forever - print ^C' 60 | print q.get() 61 | -------------------------------------------------------------------------------- /lib/exaddos/reactor.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | reactor.py 4 | 5 | Created by Thomas Mangin on 2014-02-07. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import sys 10 | 11 | from .log import log,err 12 | from .q import Queue,Empty 13 | from .http import HTTPServer 14 | from .snmp import SNMPClient 15 | from .flow import FlowServer 16 | from .container import ContainerSNMP,ContainerFlow 17 | 18 | # XXX: Look at ExaProxy Queue implementation 19 | _raising = Queue() 20 | _snmp_container = ContainerSNMP() 21 | _flow_container = ContainerFlow() 22 | 23 | _http = None 24 | _flow = None 25 | _snmp = None 26 | 27 | def setup (configuration): 28 | global _http 29 | global _flow 30 | global _snmp 31 | 32 | ip = configuration.http.host 33 | port = configuration.http.port 34 | _http = HTTPServer(configuration,_snmp_container,_flow_container) 35 | _http.add(ip,port,_raising) 36 | 37 | _snmp = SNMPClient(_snmp_container) 38 | interfaces = [_ for _ in configuration.keys() if _.isupper()] 39 | for interface in interfaces: 40 | _snmp.add(interface,configuration[interface],_raising) 41 | 42 | ip = configuration.ipfix.host 43 | port = configuration.ipfix.port 44 | _flow = FlowServer(configuration,_flow_container) 45 | _flow.add(ip,port,_raising) 46 | 47 | 48 | def run (): 49 | # we start one thread per router 50 | # therefore stopping threads for debugging can not be a on/off thing 51 | _snmp.run() 52 | 53 | _flow.run(daemon=True) 54 | _http.run(daemon=True) 55 | 56 | # _flow.run(daemon=False) 57 | # _http.run(daemon=False) 58 | 59 | # import pdb; pdb.set_trace() 60 | 61 | while True: 62 | try: 63 | exception = _raising.get() 64 | except Empty: 65 | pass 66 | except KeyboardInterrupt: 67 | break 68 | else: 69 | exc_type, exc_obj, exc_trace = exception 70 | raise exc_obj 71 | 72 | # print '.', 73 | # sys.stdout.flush() 74 | 75 | try: 76 | _http.join() 77 | if not _http.alive(): 78 | err('http server stopped / could not start, exiting') 79 | break 80 | except KeyboardInterrupt: 81 | break 82 | except Exception,e: 83 | print "exception ..." , e 84 | sys.stdout.flush() 85 | break 86 | 87 | if False: 88 | r = '' 89 | for link in _snmp_container.keys(): 90 | r += '\n%s\n' % link 91 | for k,v in _snmp_container.get(link).iteritems(): 92 | r += ' %-15s:%s\n' % (k,v) 93 | print r 94 | sys.stdout.flush() 95 | 96 | print "exiting ...." 97 | sys.stdout.flush() 98 | sys.stderr.flush() 99 | 100 | 101 | # from: http://stackoverflow.com/questions/132058/showing-the-stack-trace-from-a-running-python-application 102 | # import threading, sys, traceback 103 | 104 | # def dumpstacks(signal, frame): 105 | # id2name = dict([(th.ident, th.name) for th in threading.enumerate()]) 106 | # code = [] 107 | # for threadId, stack in sys._current_frames().items(): 108 | # code.append("\n# Thread: %s(%d)" % (id2name.get(threadId,""), threadId)) 109 | # for filename, lineno, name, line in traceback.extract_stack(stack): 110 | # code.append('File: "%s", line %d, in %s' % (filename, lineno, name)) 111 | # if line: 112 | # code.append(" %s" % (line.strip())) 113 | # print "\n".join(code) 114 | 115 | # import signal 116 | # signal.signal(signal.SIGQUIT, dumpstacks) 117 | -------------------------------------------------------------------------------- /lib/exaddos/snmp.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | snmp.py 4 | 5 | Created by Thomas Mangin on 2014-02-07. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import sys 10 | import time 11 | import random 12 | 13 | from .log import log,err 14 | from .thread import Thread 15 | from .warning import unicast,notunicast,bw 16 | 17 | # group the OID to request 18 | # http://pysnmp.sourceforge.net/docs/current/apps/sync-command-generator.html 19 | 20 | class _SNMPFactory (object): 21 | use_thread = True 22 | initialised = False 23 | 24 | correction = { 25 | 'ifHCInOctets' : 8, 26 | 'ifHCInUcastPkts' : 1, 27 | 'ifInNUcastPkts' : 1, 28 | 'ifInErrors' : 1, 29 | 'ifInDiscards' : 1, 30 | } 31 | 32 | unit = { 33 | 'ifHCInOctets' : 'bits', 34 | 'ifHCInUcastPkts' : 'pkts', 35 | 'ifInNUcastPkts' : 'pkts', 36 | 'ifInErrors' : 'pkts', 37 | 'ifInDiscards' : 'pkts', 38 | } 39 | 40 | def __init__ (self,name,interface,container,raising): 41 | self.name = name 42 | self.interface = interface 43 | self.container = container 44 | self.raising = raising 45 | self.running = None 46 | 47 | def _get (self,key): 48 | from pysnmp.entity.rfc3413.oneliner import cmdgen 49 | from pysnmp.error import PySnmpError 50 | from pysnmp.proto.rfc1905 import NoSuchInstance 51 | 52 | try: 53 | if self.interface.snmp_version == 2: 54 | errorIndication, errorStatus, errorIndex, varBinds = cmdgen.CommandGenerator().getCmd( 55 | cmdgen.CommunityData('exaddos', self.interface.snmp_password), 56 | cmdgen.UdpTransportTarget((self.interface.router, 161)), 57 | self.collection[key] 58 | ) 59 | elif self.interface.snmp_version == 3: 60 | from pysnmp.entity import config 61 | 62 | mapping_auth = { 63 | 'MD5' : config.usmHMACMD5AuthProtocol, 64 | 'SHA' : config.usmHMACSHAAuthProtocol, 65 | '' : config.usmNoAuthProtocol, 66 | } 67 | 68 | mapping_privacy = { 69 | 'DES' : config.usmDESPrivProtocol, 70 | '3DES' : config.usm3DESEDEPrivProtocol, 71 | 'AES-128' : config.usmAesCfb128Protocol, 72 | 'AES-192' : config.usmAesCfb192Protocol, 73 | 'AES-256' : config.usmAesCfb256Protocol, 74 | '' : config.usmNoPrivProtocol, 75 | } 76 | 77 | user = cmdgen.UsmUserData( 78 | self.interface.snmp_user, 79 | self.interface.snmp_auth_key, 80 | self.interface.snmp_privacy_key, 81 | authProtocol=mapping_auth[self.interface.snmp_auth_method], 82 | privProtocol=mapping_privacy[self.interface.snmp_privacy_method]) 83 | 84 | transport = cmdgen.UdpTransportTarget((self.interface.router, 161)) 85 | 86 | errorIndication, errorStatus, errorIndex, varBinds = cmdgen.CommandGenerator().getCmd( 87 | user, transport, 88 | self.collection[key] 89 | ) 90 | # cmdgen.MibVariable('.'.join(str(_) for _ in self.collection[key])) 91 | else: 92 | raise NotImplemented('Feel free to add support for this SNMP version and send us the patch - thanks') 93 | except PySnmpError: 94 | err('SNMP collection failed for %s %s' % (self.name,key)) 95 | return None 96 | 97 | if (errorIndication,errorStatus,errorIndex) == (None,0,0): 98 | result = varBinds[0][1] 99 | 100 | if isinstance(result,NoSuchInstance): 101 | err('SNMP: %s did not have %s' % (self.name,key)) 102 | sys.stderr.flush() 103 | return None 104 | 105 | try: 106 | return varBinds[0][1] 107 | except AttributeError: 108 | err('SNMP: %s did not have %s' % (self.name,key)) 109 | return None 110 | else: 111 | err('SNMP collection failed for %s %s' % (self.name,key)) 112 | return None 113 | 114 | def collect (self): 115 | result = {} 116 | 117 | for key in self.correction: 118 | value = self._get(key) 119 | if value is not None: 120 | result[key] = long(value) * self.correction[key] / self.interface.snmp_frequency 121 | else: 122 | result[key] = -1 123 | 124 | return result 125 | 126 | def serve (self): 127 | self._init() 128 | self._serve() 129 | 130 | def _init (self): 131 | from pysnmp.smi import builder 132 | 133 | mibBuilder = builder.MibBuilder().loadModules('SNMPv2-MIB', 'IF-MIB') 134 | 135 | self.collection = { 136 | 'ifHCInOctets' : mibBuilder.importSymbols('IF-MIB', 'ifHCInOctets')[0].getName() + (self.interface.snmp_index_vlan,), 137 | 'ifHCInUcastPkts' : mibBuilder.importSymbols('IF-MIB', 'ifHCInUcastPkts')[0].getName() + (self.interface.snmp_index_vlan,), 138 | 'ifInNUcastPkts' : mibBuilder.importSymbols('IF-MIB', 'ifInNUcastPkts')[0].getName() + (self.interface.snmp_index_port,), 139 | 'ifInErrors' : mibBuilder.importSymbols('IF-MIB', 'ifInErrors')[0].getName() + (self.interface.snmp_index_port,), 140 | 'ifInDiscards' : mibBuilder.importSymbols('IF-MIB', 'ifInDiscards')[0].getName() + (self.interface.snmp_index_port,), 141 | 'sysDescr' : mibBuilder.importSymbols('SNMPv2-MIB', 'sysDescr')[0].getName() + (0,), 142 | } 143 | 144 | try: 145 | self.description = str(self._get('sysDescr') or '-') 146 | self.running = True 147 | except KeyboardInterrupt: 148 | self.running = False 149 | 150 | def _serve (self): 151 | last = self.collect() 152 | 153 | values = dict(zip(last.keys(),[0] * len(last.keys()))) 154 | values['description'] = self.description 155 | values['duration'] = 0 156 | self.container.set(self.name,values) 157 | 158 | delay = random.randrange(0,self.interface.snmp_frequency*100) / 100.0 159 | # make sure we are spending the SNMP requests 160 | time.sleep(delay) 161 | 162 | err('snmp poller starting %s' % self.name) 163 | 164 | while self.running: 165 | start = time.time() 166 | 167 | try: 168 | new = self.collect() 169 | except KeyboardInterrupt: 170 | self.running = False 171 | continue 172 | 173 | values['description'] = self.description 174 | values['duration'] = float('%.2f' % max(0,time.time() - start)) 175 | 176 | for key in self.correction: 177 | if new[key] == -1: 178 | values[key] = -1 179 | else: 180 | value = new[key] - last[key] 181 | values[key] = value 182 | #formated(value,self.unit[key]) 183 | 184 | values['warning'] = True if unicast(values,self.interface) or notunicast(values,self.interface) or bw(values,self.interface) else False 185 | 186 | self.container.set(self.name,values) 187 | last = new 188 | 189 | sleep = max(0,self.interface.snmp_frequency+start-time.time()) 190 | time.sleep(sleep) 191 | 192 | err('snmp poller ended %s' % self.name) 193 | 194 | def start (self): 195 | log('starting snmp clients') 196 | sys.stdout.flush() 197 | if self.use_thread: 198 | self.snmp = Thread(self.serve,self.raising) 199 | self.snmp.daemon = True 200 | self.snmp.start() 201 | else: 202 | self.serve() 203 | 204 | def join (self): 205 | if self.snmp: 206 | self.snmp.join(0.1) 207 | 208 | class SNMPClient (object): 209 | clients = {} 210 | counter = 0 211 | 212 | def __init__ (self,container): 213 | # This will be shared among all instrance 214 | self.container = container 215 | 216 | def add (self,name,interface,raising): 217 | host = interface.router 218 | key = '%s:%d' % (host,self.counter) 219 | self.counter += 1 220 | client = _SNMPFactory(name,interface,self.container,raising) 221 | client.parent = self 222 | self.clients[key] = client 223 | 224 | def run (self): 225 | for key in self.clients: 226 | self.clients[key].start() 227 | 228 | def join (self): 229 | for key in self.clients: 230 | self.clients[key].join() 231 | 232 | def alive (self): 233 | for key in self.clients: 234 | if self.clients[key].snmp.isAlive(): 235 | return True 236 | return False 237 | -------------------------------------------------------------------------------- /lib/exaddos/thread.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | thread.py 4 | 5 | Created by Thomas Mangin on 2014-02-07. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | import sys 10 | import threading 11 | 12 | class Thread (threading.Thread): 13 | def __init__(self, target, queue): 14 | threading.Thread.__init__(self,target=target) 15 | self.queue = queue 16 | 17 | def run(self): 18 | try: 19 | threading.Thread.run(self) 20 | except Exception: 21 | self.queue.put(sys.exc_info()) 22 | -------------------------------------------------------------------------------- /lib/exaddos/warning.py: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | """ 3 | warning.py 4 | 5 | Created by Thomas Mangin on 2014-02-07. 6 | Copyright (c) 2014-2014 Exa Networks. All rights reserved. 7 | """ 8 | 9 | def unicast (information,interface): 10 | return information['ifHCInUcastPkts'] > interface.threshold_unicast 11 | 12 | def notunicast (information,interface): 13 | return information['ifInNUcastPkts'] > interface.threshold_notunicast 14 | 15 | def bw (information,interface): 16 | return information['ifHCInOctets'] > interface.threshold_bandwidth 17 | -------------------------------------------------------------------------------- /sbin/exaddos: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | dirname=`dirname $0` 4 | 5 | case $dirname in 6 | /*) 7 | cd $dirname/.. > /dev/null 8 | path=`pwd` 9 | cd - > /dev/null 10 | ;; 11 | *) 12 | cd `pwd`/$dirname/.. > /dev/null 13 | path=`pwd` 14 | cd - > /dev/null 15 | ;; 16 | esac 17 | 18 | export PYTHONPATH=$path/lib 19 | #:/usr/share/exaddos/lib/version 20 | export ETC=$path/etc/exaddos 21 | 22 | if [ "$INTERPRETER" != "" ] 23 | then 24 | INTERPRETER=`which $INTERPRETER` 25 | fi 26 | 27 | PYPY=`which pypy` 28 | #PYTHON3=`which python3` 29 | PYTHON27=`which python2.7` 30 | PYTHON26=`which python2.6` 31 | PYTHON25=`which python2.5` 32 | PYTHON24=`which python2.4` 33 | PYTHON2=`which python2` 34 | PYTHON=`which python` 35 | 36 | if [ -f "$PYPY" ] 37 | then 38 | INTERPRETER=$PYPY 39 | #elif [ -f "$PYTHON27" ] 40 | #if [ -f "$PYTHON3" ] 41 | #then 42 | # INTERPRETER=$PYTHON3 43 | elif [ -f "$PYTHON27" ] 44 | then 45 | INTERPRETER=$PYTHON27 46 | elif [ -f "$PYTHON26" ] 47 | then 48 | INTERPRETER=$PYTHON26 49 | elif [ -f "$PYTHON25" ] 50 | then 51 | INTERPRETER=$PYTHON25 52 | elif [ -f "$PYTHON24" ] 53 | then 54 | INTERPRETER=$PYTHON24 55 | elif [ -f "$PYTHON2" ] 56 | then 57 | INTERPRETER=$PYTHON2 58 | elif [ -f "$PYTHON" ] 59 | then 60 | INTERPRETER=$PYTHON 61 | else 62 | INTERPRETER=python 63 | fi 64 | 65 | APPLICATIONS=`$INTERPRETER -c "import sys,os; print ' '.join(os.path.join(_,'exaddos','application.py') for _ in sys.path if os.path.isfile('/'.join((_,'exaddos/application.py'))))"` 66 | APPLICATION=`echo $APPLICATIONS | awk '{ print $1; }'` 67 | 68 | exec $INTERPRETER -m exaddos.debug $APPLICATION $* 69 | -------------------------------------------------------------------------------- /test/.empty: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Exa-Networks/exaddos/15c5e0a5575bab4c8fbac31eeb6790eb9b710a39/test/.empty --------------------------------------------------------------------------------