├── .gitignore ├── LICENSE ├── README.md ├── _config.pl.example ├── docs └── screenshot-requests-by-status.png ├── zabbix-nginx-stats-runner.sh ├── zabbix-nginx-stats.pl └── zbx_template_nginx.xml /.gitignore: -------------------------------------------------------------------------------- 1 | log.txt 2 | _config.pl 3 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | zabbix-nginx-stats: 2 | Access log parsing and aggregation to send to zabbix server. 3 | 4 | Copyright (c) 2013, Herbert Poul (herbert.poul@gmail.com), TaPo-IT OG 5 | All rights reserved. 6 | 7 | Redistribution and use in source and binary forms, with or without modification, 8 | are permitted provided that the following conditions are met: 9 | 10 | * Redistributions of source code must retain the above copyright notice, 11 | this list of conditions and the following disclaimer. 12 | 13 | * Redistributions in binary form must reproduce the above copyright 14 | notice, this list of conditions and the following disclaimer in the 15 | documentation and/or other materials provided with the distribution. 16 | 17 | * Neither the name of SCT nor the names of its contributors may be used 18 | to endorse or promote products derived from this software without 19 | specific prior written permission. 20 | 21 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 22 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 23 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 24 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR 25 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 26 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 27 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 28 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 29 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 30 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 31 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | zabbix-nginx-stats 2 | ================== 3 | 4 | Simple script to import basic nginx statistics into zabbix. It parses the log file (currently only supports one basic format) and pushes the data into zabbix. It is written in perl with a small bash wrapper script which uses logtail. 5 | 6 | Synced values include (all since last execution): 7 | 8 | * Request Count 9 | * Request count by status code 10 | * Request Time: Average, Mean, Median, 90%, 80% and 25% lines 11 | * Upstream Response Time (Same as Request Time) 12 | * Body Bytes Sent: Average, Sum 13 | 14 | The default zabbix template has preconfigured triggers to warn on 5 errors (status code 500, 503 and 403) and graphs for request time and request count by status. 15 | 16 | Installation/Configuration: 17 | ---------------- 18 | 19 | ### checkout from github 20 | ### install dependencies 21 | 22 | * logtail2 (debian: apt-get install logtail) 23 | * Perl 24 | * Statistics::Descriptive, Date::Parse and File::Temp 25 | * for debian: install packages libstatistics-descriptive-perl libtimedate-perl 26 | 27 | ### configure logfile output: 28 | 29 | log_format timed_combined '$remote_addr $host $remote_user [$time_local] ' 30 | '"$request" $status $body_bytes_sent ' 31 | '"$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe'; 32 | 33 | access_log /var/log/nginx/access.log timed_combined; 34 | 35 | ### Import template zbx_template_nginx.xml 36 | ### configure crontab to run every 10 minutes: 37 | 38 | 8-59/10 * * * * root /home/scripts/zabbix-nginx-stats/zabbix-nginx-stats-runner.sh 39 | 40 | ### watch results coming in. 41 | 42 | ![Screenshot Requests by Status](docs/screenshot-requests-by-status.png) 43 | 44 | 45 | Changelog 46 | -------------- 47 | 48 | * 2013-07-15: Initial release 49 | 50 | Roadmap 51 | -------------- 52 | 53 | Allow pushing to multiple zabbix hosts - If you are using nginx as proxy in front of various applications it should be possible to filter these applications to get separated statistics. 54 | 55 | 56 | 57 | 58 | [![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/hpoul/zabbix-nginx-stats/trend.png)](https://bitdeli.com/free "Bitdeli Badge") 59 | 60 | -------------------------------------------------------------------------------- /_config.pl.example: -------------------------------------------------------------------------------- 1 | # Rename this file to _config.pl to customize behavior. 2 | # if you don't define a config file the default is used: $config = [ {} ]; 3 | # which means that all access log entries will be submitted to the 4 | # default zabbix host as specified in the config file. 5 | 6 | $CONFIG = [ 7 | { 8 | # example to filter out all paths starting with /zabbix 9 | filter => sub { !($_[0]->{path} =~ m|^/zabbix|); }, 10 | }, 11 | { 12 | # example to submit all access log entries for worktrail.net to another zabbix host. 13 | host=> 'worktrail.net', 14 | filter => sub { $_[0]->{hostname} eq 'worktrail.net'; }, 15 | }, 16 | ]; 17 | 18 | 19 | $DEBUG = 1; 20 | $DRYRUN = 0; 21 | $ZABBIX_SENDER = '/usr/bin/zabbix_sender'; 22 | $ZABBIX_CONF = '/etc/zabbix/zabbix_agentd.conf'; 23 | # MAXAGE is the maximum age of log entries to process, all older lines are ignored 24 | # Since this script is meant to be run every 10 minutes, make sure we don't process more logfile lines. 25 | $MAXAGE = (2+10)*60*60; 26 | 27 | 28 | -------------------------------------------------------------------------------- /docs/screenshot-requests-by-status.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/hpoul/zabbix-nginx-stats/83a2fe9794ce032ec6bb4c90b036ca05821936b0/docs/screenshot-requests-by-status.png -------------------------------------------------------------------------------- /zabbix-nginx-stats-runner.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | DAT1=/tmp/zabbix-nginx-offset.dat 4 | ACCESSLOG=/var/log/nginx/access.log 5 | 6 | dir=`dirname $0` 7 | 8 | echo "=========" >> $dir/log.txt 9 | date >> $dir/log.txt 10 | /usr/sbin/logtail2 -f$ACCESSLOG -o$DAT1 | perl $dir/zabbix-nginx-stats.pl >> $dir/log.txt 11 | 12 | -------------------------------------------------------------------------------- /zabbix-nginx-stats.pl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/perl -w 2 | 3 | # Script to parse nginx log file to calculate request counts and average request times. 4 | 5 | # log_format timed_combined '$remote_addr $host $remote_user [$time_local] ' 6 | # '"$request" $status $body_bytes_sent ' 7 | # '"$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe'; 8 | # 9 | # apt-get install libstatistics-descriptive-perl libtimedate-perl 10 | 11 | use strict; 12 | use File::Basename; 13 | use Statistics::Descriptive; 14 | use Date::Parse; 15 | use File::Temp (); 16 | 17 | 18 | use lib dirname($0); 19 | 20 | our $DEBUG = 1; 21 | our $DRYRUN = 0; 22 | our $ZABBIX_SENDER = '/usr/bin/zabbix_sender'; 23 | our $ZABBIX_CONF = '/etc/zabbix/zabbix_agentd.conf'; 24 | our $SLOW_ACCESS_MIN = 1000; 25 | # MAXAGE is the maximum age of log entries to process, all older lines are ignored 26 | # Since this script is meant to be run every 10 minutes, make sure we don't process more logfile lines. 27 | our $MAXAGE = (2+10)*60*60; 28 | 29 | our $CONFIG = [ 30 | {}, 31 | ]; 32 | 33 | eval "require '_config.pl'"; 34 | 35 | # example _config.pl file: 36 | #[ 37 | # { 38 | # #'filter' => sub { return $_[1] =~ /zabbix/; }; 39 | # filter => sub { !($_[0]->{path} =~ m|^/zabbix|); }, 40 | # }, 41 | # { 42 | # host=> 'worktrail.net', 43 | # filter => sub { $_[0]->{hostname} eq 'worktrail.net'; }, 44 | # }, 45 | #]; 46 | 47 | my $reqcount = 0; 48 | my $oldcount = 0; 49 | my $parseerrors = 0; 50 | my $request_time_total = 0; 51 | my $upstream_time_total = 0; 52 | my $statuscount = { 53 | '301' => 0, 54 | '302' => 0, 55 | '200' => 0, 56 | '404' => 0, 57 | '403' => 0, 58 | '500' => 0, 59 | '503' => 0, 60 | 61 | 'other' => 0, 62 | }; 63 | 64 | 65 | my $datafh = File::Temp->new(); 66 | print "tmpfile: " . $datafh->filename . "\n"; 67 | 68 | my $results = []; 69 | for my $cfg (@$CONFIG) { 70 | push(@$results, { 71 | s_request_time => Statistics::Descriptive::Full->new(), 72 | s_upstream_time => Statistics::Descriptive::Full->new(), 73 | body_bytes_sent => Statistics::Descriptive::Full->new(), 74 | statuscount => \%$statuscount, 75 | oldcount => 0, 76 | reqcount => 0, 77 | ignored => 0, 78 | }); 79 | } 80 | 81 | 82 | while(<>){ 83 | if ( 84 | my ( 85 | $remote_addr, 86 | $hostname, 87 | $remote_user, 88 | $time_local, 89 | $request, 90 | $status, 91 | $body_bytes_sent, 92 | $http_referer, 93 | $http_user_agent, 94 | $request_time, 95 | $upstream_response_time) = m/(\S+) (\S+) (\S+) \[(.*?)\]\s+"(.*?)" (\S+) (\S+) "(.*?)" "(.*?)" ([\d\.]+)(?: ([\d\.]+|-))?/ 96 | ) { 97 | my $l = $_; 98 | my $time = str2time($time_local); 99 | my $diff = time() - $time; 100 | 101 | my $i = 0; 102 | my ($method, $path) = split(' ', $request, 3); 103 | foreach my $cfg (@$CONFIG) { 104 | my $r = $results->[$i]; $i += 1; 105 | if (!defined $path) { 106 | $path = ''; 107 | } 108 | if (defined $cfg->{filter} && !$cfg->{filter}({ hostname => $hostname, path => $path })) { 109 | $r->{ignored} += 1; 110 | next; 111 | } 112 | if ($diff > $MAXAGE) { 113 | $r->{oldcount} += 1; 114 | } 115 | $r->{statuscount}->{defined $r->{statuscount}->{$status} ? $status : 'other'} += 1; 116 | my $reqms = int($request_time*1000); 117 | $r->{s_request_time}->add_data($reqms); 118 | if (defined $upstream_response_time && $upstream_response_time ne '-') { 119 | $r->{s_upstream_time}->add_data(int($upstream_response_time*1000)); 120 | } 121 | $r->{body_bytes_sent}->add_data($body_bytes_sent); 122 | $r->{reqcount} += 1; 123 | if ($reqms > $SLOW_ACCESS_MIN) { 124 | print "WARN SLOWREQ: $reqms: $_\n"; 125 | } 126 | } 127 | } else { 128 | $parseerrors += 1; 129 | } 130 | } 131 | 132 | sub sendstat { 133 | my ($key, $value, $cfg) = @_; 134 | 135 | my $hostparam = defined $cfg->{host} ? ' -s "'.$cfg->{host}.'" ':''; 136 | print $datafh (defined $cfg->{host} ? $cfg->{host} : '-') . " nginx[$key] $value\n"; 137 | 138 | #my $cmd = "$ZABBIX_SENDER $hostparam -c $ZABBIX_CONF -k \"nginx[$key]\" -o \"$value\" >/dev/null"; 139 | #if ($DEBUG) { 140 | # print $cmd . "\n"; 141 | #} 142 | #system $cmd if ! $DRYRUN; 143 | } 144 | sub sendstatint { 145 | my ($key, $value, $cfg) = @_; 146 | sendstat($key, int($value + 0.5), $cfg); 147 | } 148 | 149 | sub sendstatpercentile { 150 | my ($key, $obj, $percentile, $cfg) = @_; 151 | my ($val, $index) = $obj->percentile($percentile); 152 | sendstatint("${key}${percentile}", $val, $cfg); 153 | } 154 | 155 | sub printstats { 156 | my ($obj, $prefix, $cfg) = @_; 157 | if ($obj->count() == 0) { 158 | return; 159 | } 160 | sendstatint("${prefix}_avg", $obj->sum()/$obj->count(), $cfg); 161 | sendstat("${prefix}_count", $obj->count(), $cfg); 162 | sendstatint("${prefix}_mean", $obj->mean(), $cfg); 163 | sendstatpercentile("${prefix}_percentile", $obj, 25, $cfg); 164 | sendstatpercentile("${prefix}_percentile", $obj, 80, $cfg); 165 | sendstatpercentile("${prefix}_percentile", $obj, 90, $cfg); 166 | sendstatint("${prefix}_median", $obj->median(), $cfg); 167 | sendstatint("${prefix}_sum", $obj->sum(), $cfg); 168 | } 169 | sub printbytestats { 170 | my ($obj, $prefix, $cfg) = @_; 171 | if ($obj->count() == 0) { 172 | return; 173 | } 174 | sendstatint("${prefix}_avg", $obj->sum()/$obj->count(), $cfg); 175 | #sendstat("${prefix}_count", $obj->count(), $cfg); 176 | sendstatint("${prefix}_sum", $obj->sum(), $cfg); 177 | } 178 | 179 | my $j = 0; 180 | foreach my $cfg (@$CONFIG) { 181 | my $r = $results->[$j]; $j++; 182 | sendstat('oldcount', $r->{oldcount}, $cfg); 183 | sendstat('requestcount', $r->{reqcount}, $cfg); 184 | sendstat('ignored', $r->{ignored}, $cfg); 185 | printstats($r->{s_request_time}, 'request_time', $cfg); 186 | printstats($r->{s_upstream_time}, 'upstream_time', $cfg); 187 | printbytestats($r->{body_bytes_sent}, 'body_bytes_sent', $cfg); 188 | sendstat("parseerrors", $parseerrors, $cfg); 189 | for my $status (keys %{$r->{statuscount}}) { 190 | sendstat("status_$status", $statuscount->{$status}, $cfg); 191 | } 192 | } 193 | 194 | 195 | my $cmd = "$ZABBIX_SENDER -vv -c $ZABBIX_CONF -i " . $datafh->filename() . " 2>&1"; 196 | print $cmd."\n"; 197 | system "cp ".$datafh->filename()." /tmp/test.txt"; 198 | system $cmd unless $DRYRUN; 199 | 200 | -------------------------------------------------------------------------------- /zbx_template_nginx.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 2.0 4 | 2013-07-26T10:17:46Z 5 | 6 | 7 | Templates 8 | 9 | 10 | 11 | 1321 | 1322 | 1323 | 1324 | {Nginx:nginx[parseerrors].last(0)}>0 1325 | Logfile had lines which lead to parseerrors 1326 | 1327 | 0 1328 | 2 1329 | 1330 | 0 1331 | 1332 | 1333 | 1334 | {Nginx:nginx[oldcount].last(0)}>0 1335 | Old Logfile lines were ignore 1336 | 1337 | 0 1338 | 2 1339 | 1340 | 0 1341 | 1342 | 1343 | 1344 | {Nginx:nginx[status_500].last(0)} + {Nginx:nginx[status_503].last(0)} + {Nginx:nginx[status_403].last(0)}>4 1345 | Too many errors: 500, 503 and 403 1346 | 1347 | 0 1348 | 2 1349 | 1350 | 0 1351 | 1352 | 1353 | 1354 | 1355 | 1356 | Requests by Status 1357 | 900 1358 | 400 1359 | 0.0000 1360 | 100.0000 1361 | 1 1362 | 1 1363 | 1 1364 | 1 1365 | 0 1366 | 0.0000 1367 | 0.0000 1368 | 0 1369 | 0 1370 | 0 1371 | 0 1372 | 1373 | 1374 | 1 1375 | 0 1376 | 0000C8 1377 | 0 1378 | 2 1379 | 0 1380 | 1381 | Nginx 1382 | nginx[status_404] 1383 | 1384 | 1385 | 1386 | 6 1387 | 0 1388 | C8C8C8 1389 | 0 1390 | 2 1391 | 0 1392 | 1393 | Nginx 1394 | nginx[status_301] 1395 | 1396 | 1397 | 1398 | 0 1399 | 0 1400 | 00CC00 1401 | 0 1402 | 2 1403 | 0 1404 | 1405 | Nginx 1406 | nginx[status_200] 1407 | 1408 | 1409 | 1410 | 2 1411 | 0 1412 | 770000 1413 | 0 1414 | 2 1415 | 0 1416 | 1417 | Nginx 1418 | nginx[status_500] 1419 | 1420 | 1421 | 1422 | 5 1423 | 0 1424 | 00C8C8 1425 | 0 1426 | 2 1427 | 0 1428 | 1429 | Nginx 1430 | nginx[status_other] 1431 | 1432 | 1433 | 1434 | 4 1435 | 0 1436 | C800C8 1437 | 0 1438 | 2 1439 | 0 1440 | 1441 | Nginx 1442 | nginx[status_503] 1443 | 1444 | 1445 | 1446 | 3 1447 | 0 1448 | CC0000 1449 | 0 1450 | 2 1451 | 0 1452 | 1453 | Nginx 1454 | nginx[status_403] 1455 | 1456 | 1457 | 1458 | 7 1459 | 0 1460 | DDDDDD 1461 | 0 1462 | 2 1463 | 0 1464 | 1465 | Nginx 1466 | nginx[status_302] 1467 | 1468 | 1469 | 1470 | 1471 | 1472 | Request Time 1473 | 900 1474 | 300 1475 | 0.0000 1476 | 100.0000 1477 | 1 1478 | 1 1479 | 0 1480 | 1 1481 | 0 1482 | 0.0000 1483 | 0.0000 1484 | 0 1485 | 0 1486 | 0 1487 | 0 1488 | 1489 | 1490 | 1 1491 | 0 1492 | 00C800 1493 | 0 1494 | 2 1495 | 0 1496 | 1497 | Nginx 1498 | nginx[upstream_time_avg] 1499 | 1500 | 1501 | 1502 | 0 1503 | 0 1504 | C80000 1505 | 0 1506 | 2 1507 | 0 1508 | 1509 | Nginx 1510 | nginx[request_time_avg] 1511 | 1512 | 1513 | 1514 | 1515 | 1516 | 1517 | --------------------------------------------------------------------------------