mirror of
https://github.com/manuelkasper/AS-Stats.git
synced 2025-02-20 11:44:12 +08:00
Merge netflow/sflow-asstatd.pl into one script
Contributed by Wouter de Jong
This commit is contained in:
Manuel Kasper
parent
0c971e3600
commit
1da6825579
@ -1,3 +1,11 @@
|
||||
## 1.5
|
||||
|
||||
* Merged netflow-asstatd.pl and sflow-asstatd.pl into one script so
|
||||
that it can handle NetFlow and sFlow sources concurrently
|
||||
(contributed by Wouter de Jong). Please note the following changes:
|
||||
* The sampling rate command line parameter (-s) has been removed. Instead, the sampling rate must now be specified for each link in the knownlinks file to avoid confusion with prior defaults. **If you're using NetFlow without sampling, you need to add the sampling rate 1 to each link.**
|
||||
* The command line parameter to set the sFlow listen port has been changed to -P to avoid a clash with the NetFlow port parameter (-p).
|
||||
|
||||
## 1.43
|
||||
|
||||
* Add v6 data sources to add_ds_proc.pl
|
||||
|
||||
37
README.md
37
README.md
@ -1,13 +1,13 @@
|
||||
AS-Stats v1.43 (2013-12-06)
|
||||
AS-Stats v1.5b (2014-01-xx)
|
||||
===========================
|
||||
|
||||
A simple tool to generate per-AS traffic graphs from NetFlow/sFlow records
|
||||
A simple tool to generate per-AS traffic graphs from NetFlow/sFlow records
|
||||
by Manuel Kasper <mk@neon1.net> for Monzoon Networks AG
|
||||
|
||||
How it works
|
||||
------------
|
||||
|
||||
A Perl script (netflow-asstatd.pl) collects NetFlow v8/v9 AS aggregation records
|
||||
A Perl script (asstatd.pl) collects NetFlow v8/v9 AS aggregation records
|
||||
or sFlow v5 samples from one or more routers. It caches them for about a
|
||||
minute (to prevent excessive writes to RRD files), identifies the link that
|
||||
each record refers to (by means of the SNMP in/out interface index), maps it
|
||||
@ -40,11 +40,7 @@ Prerequisites
|
||||
|
||||
Installation
|
||||
------------
|
||||
In the instructions below, "xx-asstatd.pl" refers to either netflow-asstatd.pl
|
||||
or sflow-asstatd.pl, depending on whether your routers generate NetFlow or
|
||||
sFlow data.
|
||||
|
||||
- Copy the perl scripts xx-asstatd.pl and rrd-extractstats.pl to the
|
||||
- Copy the perl scripts asstatd.pl and rrd-extractstats.pl to the
|
||||
machine that will collect NetFlow/sFlow records
|
||||
|
||||
- Create a "known links" file with the following information about each
|
||||
@ -57,6 +53,7 @@ sFlow data.
|
||||
internally (e.g. for RRD DS names)
|
||||
- a human-readable description (will appear in the generated graphs)
|
||||
- a color code for the graphs (HTML style, 6 hex digits)
|
||||
- the sampling rate (or 1 if you're not using sampling on the router)
|
||||
|
||||
See the example file provided (knownlinks) for the format.
|
||||
__Important: you must use tabs, not spaces, to separate fields!__
|
||||
@ -67,21 +64,19 @@ sFlow data.
|
||||
more efficient storage of RRD files (one directory per lower byte of
|
||||
AS number, in hex).
|
||||
|
||||
- Start xx-asstatd.pl in the background (or, better yet, write a
|
||||
- Start asstatd.pl in the background (or, better yet, write a
|
||||
startup script for your operating system to automatically start
|
||||
xx-asstatd.pl on boot):
|
||||
asstatd.pl on boot):
|
||||
|
||||
`nohup xx-asstatd.pl -r /path/to/rrd/dir -k /path/to/knownlinks &`
|
||||
`nohup asstatd.pl -r /path/to/rrd/dir -k /path/to/knownlinks &`
|
||||
|
||||
By default, netflow-asstatd.pl will listen on port 9000 (UDP) for NetFlow
|
||||
datagrams, and sflow-asstatd.pl will listen on port 6343 (UDP) for sFlow
|
||||
datagrams. Use the -p option if you want to change that.
|
||||
If you use sampled NetFlow or sFlow, set the sampling rate with the -s
|
||||
option.
|
||||
sflow-asstatd.pl also needs you to specify your own AS number with the -a
|
||||
By default, asstatd.pl will listen on port 9000 (UDP) for NetFlow
|
||||
datagrams, and on port 6343 (UDP) for sFlow datagrams. Use the -p/-P options
|
||||
if you want to change that (use 0 as the port number to disable either protocol).
|
||||
For sFlow, you also need to specify your own AS number with the -a
|
||||
option for accurate classification of inbound and outbound traffic.
|
||||
It's a good idea to make sure only UDP datagrams from your trusted routers
|
||||
will reach the machine running xx-asstatd.pl (firewall etc.).
|
||||
will reach the machine running asstatd.pl (firewall etc.).
|
||||
|
||||
- NetFlow only:
|
||||
Have your router(s) send NetFlow v8 or v9 AS aggregation records to
|
||||
@ -254,7 +249,7 @@ sFlow data.
|
||||
|
||||
- Wait 1-2 minutes. You should then see new RRD files popping up in the
|
||||
directory that you defined/created earlier on. If not, make sure that
|
||||
xx-asstatd.pl is running, not spewing out any error messages, and that
|
||||
asstatd.pl is running, not spewing out any error messages, and that
|
||||
the NetFlow/sFlow datagrams are actually reaching your machine (tcpdump...).
|
||||
|
||||
- Add a cronjob to run the following command every hour:
|
||||
@ -285,7 +280,7 @@ Adding a new link involves adding two new data sources to all RRD files.
|
||||
This is a bit of a PITA since RRDtool itself doesn't provide a command to do
|
||||
that. A simple (but slow) Perl script that is meant to be used with RRDtool's
|
||||
XML dump/restore feature is provided (add_ds_proc.pl, add_ds.sh). Note that
|
||||
netflow-asstatd.pl should be stopped while modifying RRD files, to avoid
|
||||
asstatd.pl should be stopped while modifying RRD files, to avoid
|
||||
breaking them with concurrent modifications.
|
||||
|
||||
|
||||
@ -299,7 +294,7 @@ By default, the created RRDs keep data as follows:
|
||||
* 1 year at 1 day resolution
|
||||
|
||||
If you want to change that, modify the getrrdfile() function in
|
||||
xx-asstatd.pl and delete any old RRD files.
|
||||
asstatd.pl and delete any old RRD files.
|
||||
|
||||
|
||||
To do
|
||||
|
||||
228
bin/netflow-asstatd.pl → bin/asstatd.pl
Executable file → Normal file
228
bin/netflow-asstatd.pl → bin/asstatd.pl
Executable file → Normal file
@ -6,6 +6,7 @@
|
||||
# cli params/rrd storage/sampling mods Steve Colam <steve@colam.co.uk>
|
||||
|
||||
use strict;
|
||||
use IO::Select;
|
||||
use IO::Socket;
|
||||
use RRDs;
|
||||
use Getopt::Std;
|
||||
@ -13,18 +14,18 @@ use Getopt::Std;
|
||||
my %knownlinks;
|
||||
my %link_samplingrates;
|
||||
|
||||
my $samplingrate = 1; # rate for sampled NetFlow (or = 1 for unsampled)
|
||||
|
||||
my $ascache = {};
|
||||
my $ascache_lastflush = 0;
|
||||
my $ascache_flush_interval = 25;
|
||||
my $ascache_flush_number = 0;
|
||||
|
||||
my $server_port = 9000;
|
||||
my $MAXREAD = 8192;
|
||||
my $childrunning = 0;
|
||||
|
||||
# NetFlow
|
||||
my $server_port = 9000;
|
||||
my $v5_header_len = 24;
|
||||
my $v5_flowrec_len = 48;
|
||||
my $childrunning = 0;
|
||||
my $v8_header_len = 28;
|
||||
my $v8_flowrec_len = 28;
|
||||
my $v9_header_len = 20;
|
||||
@ -32,33 +33,45 @@ my $v9_templates = {};
|
||||
my $v10_header_len = 16;
|
||||
my $v10_templates = {};
|
||||
|
||||
use vars qw/ %opt /;
|
||||
getopts('r:p:k:s:', \%opt);
|
||||
# sFlow
|
||||
my $sflow_server_port = 6343;
|
||||
|
||||
my $usage = "$0 [-rpks]\n".
|
||||
use vars qw/ %opt /;
|
||||
getopts('r:p:P:k:a:', \%opt);
|
||||
|
||||
my $usage = "$0 [-rpPka]\n".
|
||||
"\t-r <path to RRD files>\n".
|
||||
"\t(-p <UDP listen port - default $server_port>)\n".
|
||||
"\t(-p <NetFlow UDP listen port - default $server_port, use 0 to disable NetFlow)\n".
|
||||
"\t(-P <sFlow UDP listen port - default $sflow_server_port, use 0 to disable sFlow)\n".
|
||||
"\t-k <path to known links file>\n".
|
||||
"\t(-s <sampling rate - default $samplingrate>)\n";
|
||||
"\t-a <your own AS number> - only required for sFlow\n";
|
||||
|
||||
my $rrdpath = $opt{'r'};
|
||||
my $knownlinksfile = $opt{'k'};
|
||||
my $myas = $opt{'a'};
|
||||
|
||||
die("$usage") if (!defined($rrdpath) || !defined($knownlinksfile));
|
||||
|
||||
die("$rrdpath does not exist or is not a directory\n") if ! -d $rrdpath;
|
||||
die("$knownlinksfile does not exist or is not a file\n") if ! -f $knownlinksfile;
|
||||
|
||||
if (defined($opt{'s'})) {
|
||||
$samplingrate = $opt{'s'};
|
||||
die("Sampling rate is non numeric\n") if $samplingrate !~ /^[0-9]+$/;
|
||||
}
|
||||
|
||||
if (defined($opt{'p'})) {
|
||||
$server_port = $opt{'p'};
|
||||
die("Server port is non numeric\n") if $server_port !~ /^[0-9]+$/;
|
||||
die("NetFlow server port is non numeric\n") if $server_port !~ /^[0-9]+$/;
|
||||
}
|
||||
|
||||
if (defined($opt{'P'})) {
|
||||
$sflow_server_port = $opt{'P'};
|
||||
die("sFlow server port is non numeric\n") if $sflow_server_port !~ /^[0-9]+$/;
|
||||
}
|
||||
|
||||
if ($sflow_server_port == $server_port) {
|
||||
die("sFlow server port can't be the same as NetFlow server port\n");
|
||||
}
|
||||
|
||||
die("Your own AS number is non numeric\n") if ($sflow_server_port > 0 && $myas !~ /^[0-9]+$/);
|
||||
|
||||
|
||||
# reap dead children
|
||||
$SIG{CHLD} = \&REAPER;
|
||||
$SIG{TERM} = \&TERM;
|
||||
@ -79,33 +92,56 @@ sub TERM {
|
||||
# read known links file
|
||||
read_knownlinks();
|
||||
|
||||
my ($lsn_nflow, $lsn_sflow);
|
||||
my $sel = IO::Select->new();
|
||||
|
||||
# prepare to listen for NetFlow UDP packets
|
||||
my $server = IO::Socket::INET->new(LocalPort => $server_port, Proto => "udp")
|
||||
or die "Couldn't be a udp server on port $server_port : $@\n";
|
||||
if ($server_port > 0) {
|
||||
$lsn_nflow = IO::Socket::INET->new(LocalPort => $server_port, Proto => "udp")
|
||||
or die "Couldn't be a NetFlow UDP server on port $server_port : $@\n";
|
||||
$sel->add($lsn_nflow);
|
||||
}
|
||||
# prepare to listen for sFlow UDP packets
|
||||
if ($sflow_server_port > 0) {
|
||||
require Net::sFlow;
|
||||
$lsn_sflow = IO::Socket::INET->new(LocalPort => $sflow_server_port, Proto => "udp")
|
||||
or die "Couldn't be a sFlow UDP server on port $sflow_server_port : $@\n";
|
||||
$sel->add($lsn_sflow);
|
||||
}
|
||||
|
||||
my ($him,$datagram,$flags);
|
||||
|
||||
# main NetFlow datagram receive loop
|
||||
# main datagram receive loop
|
||||
while (1) {
|
||||
$him = $server->recv($datagram, $MAXREAD);
|
||||
next if (!$him);
|
||||
|
||||
my ($port, $ipaddr) = sockaddr_in($server->peername);
|
||||
|
||||
my ($version) = unpack("n", $datagram);
|
||||
|
||||
if ($version == 5) {
|
||||
parse_netflow_v5($datagram, $ipaddr);
|
||||
} elsif ($version == 8) {
|
||||
parse_netflow_v8($datagram, $ipaddr);
|
||||
} elsif ($version == 9) {
|
||||
parse_netflow_v9($datagram, $ipaddr);
|
||||
} elsif ($version == 10) {
|
||||
parse_netflow_v10($datagram, $ipaddr);
|
||||
} else {
|
||||
print "unknown NetFlow version: $version\n";
|
||||
while (my @ready = $sel->can_read) {
|
||||
foreach my $server (@ready) {
|
||||
$him = $server->recv($datagram, $MAXREAD);
|
||||
next if (!$him);
|
||||
|
||||
my ($port, $ipaddr) = sockaddr_in($server->peername);
|
||||
|
||||
if (defined($lsn_nflow) && $server == $lsn_nflow) {
|
||||
my ($version) = unpack("n", $datagram);
|
||||
|
||||
if ($version == 5) {
|
||||
parse_netflow_v5($datagram, $ipaddr);
|
||||
} elsif ($version == 8) {
|
||||
parse_netflow_v8($datagram, $ipaddr);
|
||||
} elsif ($version == 9) {
|
||||
parse_netflow_v9($datagram, $ipaddr);
|
||||
} elsif ($version == 10) {
|
||||
parse_netflow_v10($datagram, $ipaddr);
|
||||
} else {
|
||||
print "unknown NetFlow version: $version\n";
|
||||
}
|
||||
}
|
||||
elsif (defined($lsn_sflow) && $server == $lsn_sflow) {
|
||||
parse_sflow($datagram, $ipaddr);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sub parse_netflow_v5 {
|
||||
my $datagram = shift;
|
||||
my $ipaddr = shift;
|
||||
@ -119,12 +155,11 @@ sub parse_netflow_v5 {
|
||||
for (my $i = 0; $i < $count; $i++) {
|
||||
my $flowrec = substr($datagram, $v5_header_len + ($i*$v5_flowrec_len), $v5_flowrec_len);
|
||||
my @flowdata = unpack("NNNnnNNNNnnccccnnccN", $flowrec);
|
||||
print "ipaddr: " . inet_ntoa($ipaddr) . " octets: $flowdata[6] srcas: $flowdata[15] dstas: $flowdata[16] in: $flowdata[3] out: $flowdata[4] 4 \n";
|
||||
handleflow($ipaddr, $flowdata[6], $flowdata[15], $flowdata[16], $flowdata[3], $flowdata[4], 4);
|
||||
#print "ipaddr: " . inet_ntoa($ipaddr) . " octets: $flowdata[6] srcas: $flowdata[15] dstas: $flowdata[16] in: $flowdata[3] out: $flowdata[4] 4 \n";
|
||||
handleflow($ipaddr, $flowdata[6], $flowdata[15], $flowdata[16], $flowdata[3], $flowdata[4], 4, 'netflow');
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
sub parse_netflow_v8 {
|
||||
my $datagram = shift;
|
||||
my $ipaddr = shift;
|
||||
@ -143,7 +178,8 @@ sub parse_netflow_v8 {
|
||||
for (my $i = 0; $i < $count; $i++) {
|
||||
my $flowrec = substr($datagram, $v8_header_len + ($i*$v8_flowrec_len), $v8_flowrec_len);
|
||||
my @flowdata = unpack("NNNNNnnnn", $flowrec);
|
||||
handleflow($ipaddr, $flowdata[2], $flowdata[5], $flowdata[6], $flowdata[7], $flowdata[8], 4);
|
||||
#print "ipaddr: " . inet_ntoa($ipaddr) . " octets: $flowdata[2] srcas: $flowdata[5] dstas: $flowdata[6] in: $flowdata[7] out: $flowdata[8] 4 \n";
|
||||
handleflow($ipaddr, $flowdata[2], $flowdata[5], $flowdata[6], $flowdata[7], $flowdata[8], 4, 'netflow');
|
||||
}
|
||||
}
|
||||
|
||||
@ -280,7 +316,7 @@ sub parse_netflow_v9_data_flowset {
|
||||
}
|
||||
|
||||
if (defined($srcas) && defined($dstas) && defined($snmpin) && defined($snmpout)) {
|
||||
handleflow($ipaddr, $inoctets + $outoctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion);
|
||||
handleflow($ipaddr, $inoctets + $outoctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion, 'netflow');
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -421,19 +457,103 @@ sub parse_netflow_v10_data_flowset {
|
||||
}
|
||||
|
||||
if (defined($srcas) && defined($dstas) && defined($snmpin) && defined($snmpout)) {
|
||||
handleflow($ipaddr, $inoctets + $outoctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion);
|
||||
handleflow($ipaddr, $inoctets + $outoctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion, 'netflow');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sub parse_sflow {
|
||||
my $datagram = shift;
|
||||
my $ipaddr = shift;
|
||||
|
||||
# decode the sFlow packet
|
||||
my ($sFlowDatagramRef, $sFlowSamplesRef, $errorsRef) = Net::sFlow::decode($datagram);
|
||||
|
||||
if ($sFlowDatagramRef->{'sFlowVersion'} != 5) {
|
||||
print "Warning: non-v5 packet received - not supported\n";
|
||||
return;
|
||||
}
|
||||
|
||||
# use agent IP if available (in case of proxy)
|
||||
if ($sFlowDatagramRef->{'AgentIp'}) {
|
||||
$ipaddr = inet_aton($sFlowDatagramRef->{'AgentIp'});
|
||||
}
|
||||
|
||||
foreach my $sFlowSample (@{$sFlowSamplesRef}) {
|
||||
my $ipversion = 4;
|
||||
|
||||
# only process standard structures
|
||||
next if ($sFlowSample->{'sampleTypeEnterprise'} != 0);
|
||||
|
||||
# only process normal flow samples
|
||||
next if ($sFlowSample->{'sampleTypeFormat'} != 1);
|
||||
|
||||
my $snmpin = $sFlowSample->{'inputInterface'};
|
||||
my $snmpout = $sFlowSample->{'outputInterface'};
|
||||
|
||||
if ($snmpin >= 1073741823 || $snmpout >= 1073741823) {
|
||||
# invalid interface index - could be dropped packet or internal
|
||||
# (routing protocol, management etc.)
|
||||
#print "Invalid interface index $snmpin/$snmpout\n";
|
||||
next;
|
||||
}
|
||||
|
||||
my $noctets;
|
||||
if ($sFlowSample->{'IPv4Packetlength'}) {
|
||||
$noctets = $sFlowSample->{'IPv4Packetlength'};
|
||||
} elsif ($sFlowSample->{'IPv6Packetlength'}) {
|
||||
$noctets = $sFlowSample->{'IPv6Packetlength'};
|
||||
$ipversion = 6;
|
||||
} else {
|
||||
$noctets = $sFlowSample->{'HeaderFrameLength'} - 14;
|
||||
|
||||
# make one more attempt at figuring out the IP version
|
||||
if ((defined($sFlowSample->{'GatewayIpVersionNextHopRouter'}) &&
|
||||
$sFlowSample->{'GatewayIpVersionNextHopRouter'} == 2) ||
|
||||
(defined($sFlowSample->{'HeaderType'}) && $sFlowSample->{'HeaderType'} eq '86dd')) {
|
||||
$ipversion = 6;
|
||||
}
|
||||
}
|
||||
|
||||
my $srcas = 0;
|
||||
my $dstas = 0;
|
||||
|
||||
if ($sFlowSample->{'GatewayAsSource'}) {
|
||||
$srcas = $sFlowSample->{'GatewayAsSource'};
|
||||
}
|
||||
if ($sFlowSample->{'GatewayDestAsPaths'}) {
|
||||
$dstas = pop(@{$sFlowSample->{'GatewayDestAsPaths'}->[0]->{'AsPath'}});
|
||||
if (!$dstas) {
|
||||
$dstas = 0;
|
||||
}
|
||||
}
|
||||
|
||||
# Outbound packets have our AS number as the source (GatewayAsSource),
|
||||
# while inbound packets have 0 as the destination (empty AsPath).
|
||||
# Transit packets have "foreign" AS numbers for both source and
|
||||
# destination (handleflow() currently deals with those by counting
|
||||
# them twice; once for input and once for output)
|
||||
|
||||
# substitute 0 for own AS number
|
||||
if ($srcas == $myas) {
|
||||
$srcas = 0;
|
||||
}
|
||||
if ($dstas == $myas) {
|
||||
$dstas = 0;
|
||||
}
|
||||
|
||||
handleflow($ipaddr, $noctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion, 'sflow');
|
||||
}
|
||||
}
|
||||
|
||||
sub handleflow {
|
||||
my ($routerip, $noctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion) = @_;
|
||||
|
||||
if ($srcas == 0 && $dstas == 0) {
|
||||
# don't care about internal traffic
|
||||
# don't care about internal traffic
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
#print "$srcas => $dstas ($noctets octets, version $ipversion, snmpin $snmpin, snmpout $snmpout)\n";
|
||||
|
||||
# determine direction and interface alias name (if known)
|
||||
@ -451,7 +571,7 @@ sub handleflow {
|
||||
$ifalias = $knownlinks{inet_ntoa($routerip) . '_' . $snmpin};
|
||||
} else {
|
||||
handleflow($routerip, $noctets, $srcas, 0, $snmpin, $snmpout, $ipversion);
|
||||
handleflow($routerip, $noctets, 0, $dstas, $snmpin, $snmpout, $ipversion);
|
||||
handleflow($routerip, $noctets, 0, $dstas, $snmpin, $snmpout, $ipversion);
|
||||
return;
|
||||
}
|
||||
|
||||
@ -485,7 +605,6 @@ sub handleflow {
|
||||
}
|
||||
|
||||
sub flush_cache {
|
||||
|
||||
if ($childrunning || ((time - $ascache_lastflush) < $ascache_flush_interval)) {
|
||||
# can't/don't want to flush cache right now
|
||||
return;
|
||||
@ -522,12 +641,8 @@ sub flush_cache {
|
||||
|
||||
my $tag = $dsname;
|
||||
$tag =~ s/(_v6)?_(in|out)$//;
|
||||
my $cursamplingrate = $samplingrate;
|
||||
my $cursamplingrate = $link_samplingrates{$tag};
|
||||
|
||||
if ($link_samplingrates{$tag}) {
|
||||
$cursamplingrate = $link_samplingrates{$tag};
|
||||
}
|
||||
|
||||
push(@templatearg, $dsname);
|
||||
push(@args, $value * $cursamplingrate);
|
||||
}
|
||||
@ -550,7 +665,7 @@ sub getrrdfile {
|
||||
my $as = shift;
|
||||
my $startts = shift;
|
||||
$startts--;
|
||||
|
||||
|
||||
# we create 256 directories and store RRD files based on the lower
|
||||
# 8 bytes of the AS number
|
||||
my $dirname = "$rrdpath/" . sprintf("%02x", $as % 256);
|
||||
@ -593,17 +708,19 @@ sub getrrdfile {
|
||||
sub read_knownlinks {
|
||||
my %knownlinks_tmp;
|
||||
my %link_samplingrates_tmp;
|
||||
open(KLFILE, $knownlinksfile) or die("Cannot open $knownlinksfile!");
|
||||
open(KLFILE, $knownlinksfile) or die("Cannot open $knownlinksfile: $!");
|
||||
while (<KLFILE>) {
|
||||
chomp;
|
||||
next if (/(^\s*#)|(^\s*$)/); # empty line or comment
|
||||
|
||||
my ($routerip,$ifindex,$tag,$descr,$color,$samplingrate) = split(/\t+/);
|
||||
my ($routerip,$ifindex,$tag,$descr,$color,$linksamplingrate) = split(/\t+/);
|
||||
$knownlinks_tmp{"${routerip}_${ifindex}"} = $tag;
|
||||
|
||||
if ($samplingrate) {
|
||||
$link_samplingrates_tmp{$tag} = $samplingrate;
|
||||
unless(defined($linksamplingrate) && $linksamplingrate =~ /^\d+$/) {
|
||||
die("ERROR: No samplingrate for ".$routerip."\n");
|
||||
}
|
||||
|
||||
$link_samplingrates_tmp{$tag} = $linksamplingrate;
|
||||
}
|
||||
close(KLFILE);
|
||||
|
||||
@ -611,3 +728,4 @@ sub read_knownlinks {
|
||||
%link_samplingrates = %link_samplingrates_tmp;
|
||||
return;
|
||||
}
|
||||
|
||||
@ -1,356 +0,0 @@
|
||||
#!/usr/bin/perl -w
|
||||
#
|
||||
# $Id$
|
||||
#
|
||||
# written by Manuel Kasper <mk@neon1.net> for Monzoon Networks AG
|
||||
|
||||
use strict;
|
||||
use Net::sFlow;
|
||||
use IO::Socket;
|
||||
use RRDs;
|
||||
use Getopt::Std;
|
||||
|
||||
my %knownlinks;
|
||||
my %link_samplingrates;
|
||||
|
||||
my $samplingrate = 512;
|
||||
|
||||
my $ascache = {};
|
||||
my $ascache_lastflush = 0;
|
||||
my $ascache_flush_interval = 25;
|
||||
my $ascache_flush_number = 0;
|
||||
|
||||
my $server_port = 6343;
|
||||
my $MAXREAD = 8192;
|
||||
my $header_len = 28;
|
||||
my $flowrec_len = 28;
|
||||
my $childrunning = 0;
|
||||
|
||||
use vars qw/ %opt /;
|
||||
getopts('r:p:k:a:s:', \%opt);
|
||||
|
||||
my $usage = "$0 [-rpkas]\n".
|
||||
"\t-r <path to RRD files>\n".
|
||||
"\t(-p <UDP listen port - default $server_port>)\n".
|
||||
"\t-k <path to known links file>\n".
|
||||
"\t-a <your own AS number>\n".
|
||||
"\t(-s <sampling rate - default $samplingrate>)\n";
|
||||
|
||||
my $rrdpath = $opt{'r'};
|
||||
my $knownlinksfile = $opt{'k'};
|
||||
my $myas = $opt{'a'};
|
||||
|
||||
die("$usage") if (!defined($rrdpath) || !defined($knownlinksfile) || !defined($myas));
|
||||
|
||||
die("$rrdpath does not exist or is not a directory\n") if ! -d $rrdpath;
|
||||
die("$knownlinksfile does not exist or is not a file\n") if ! -f $knownlinksfile;
|
||||
die("Your own AS number is non numeric\n") if ($myas !~ /^[0-9]+$/);
|
||||
|
||||
if (defined($opt{'s'})) {
|
||||
$samplingrate = $opt{'s'};
|
||||
die("Sampling rate is non numeric\n") if $samplingrate !~ /^[0-9]+$/;
|
||||
}
|
||||
|
||||
if (defined($opt{'p'})) {
|
||||
$server_port = $opt{'p'};
|
||||
die("Server port is non numeric\n") if $server_port !~ /^[0-9]+$/;
|
||||
}
|
||||
|
||||
# reap dead children
|
||||
$SIG{CHLD} = \&REAPER;
|
||||
$SIG{TERM} = \&TERM;
|
||||
$SIG{INT} = \&TERM;
|
||||
$SIG{HUP} = \&read_knownlinks;
|
||||
|
||||
sub REAPER {
|
||||
wait;
|
||||
$childrunning = 0;
|
||||
$SIG{CHLD} = \&REAPER;
|
||||
}
|
||||
|
||||
sub TERM {
|
||||
print "SIGTERM received\n";
|
||||
exit 0;
|
||||
}
|
||||
|
||||
# read known links file
|
||||
read_knownlinks();
|
||||
|
||||
# prepare to listen for sFlow UDP packets
|
||||
my $server = IO::Socket::INET->new(LocalPort => $server_port, Proto => "udp")
|
||||
or die "Couldn't be a udp server on port $server_port : $@\n";
|
||||
|
||||
my ($him,$datagram,$flags);
|
||||
|
||||
# main sFlow datagram receive loop
|
||||
while (1) {
|
||||
$him = $server->recv($datagram, $MAXREAD);
|
||||
next if (!$him);
|
||||
|
||||
my ($port, $ipaddr) = sockaddr_in($server->peername);
|
||||
|
||||
# decode the sFlow packet
|
||||
my ($sFlowDatagramRef, $sFlowSamplesRef, $errorsRef) = Net::sFlow::decode($datagram);
|
||||
|
||||
if ($sFlowDatagramRef->{'sFlowVersion'} != 5) {
|
||||
print "Warning: non-v5 packet received - not supported\n";
|
||||
next;
|
||||
}
|
||||
|
||||
# use agent IP if available (in case of proxy)
|
||||
if ($sFlowDatagramRef->{'AgentIp'}) {
|
||||
$ipaddr = inet_aton($sFlowDatagramRef->{'AgentIp'});
|
||||
}
|
||||
|
||||
foreach my $sFlowSample (@{$sFlowSamplesRef}) {
|
||||
my $ipversion = 4;
|
||||
|
||||
# only process standard structures
|
||||
next if ($sFlowSample->{'sampleTypeEnterprise'} != 0);
|
||||
|
||||
# only process normal flow samples
|
||||
next if ($sFlowSample->{'sampleTypeFormat'} != 1);
|
||||
|
||||
my $snmpin = $sFlowSample->{'inputInterface'};
|
||||
my $snmpout = $sFlowSample->{'outputInterface'};
|
||||
|
||||
if ($snmpin >= 1073741823 || $snmpout >= 1073741823) {
|
||||
# invalid interface index - could be dropped packet or internal
|
||||
# (routing protocol, management etc.)
|
||||
#print "Invalid interface index $snmpin/$snmpout\n";
|
||||
next;
|
||||
}
|
||||
|
||||
my $noctets;
|
||||
if ($sFlowSample->{'IPv4Packetlength'}) {
|
||||
$noctets = $sFlowSample->{'IPv4Packetlength'};
|
||||
} elsif ($sFlowSample->{'IPv6Packetlength'}) {
|
||||
$noctets = $sFlowSample->{'IPv6Packetlength'};
|
||||
$ipversion = 6;
|
||||
} else {
|
||||
$noctets = $sFlowSample->{'HeaderFrameLength'} - 14;
|
||||
|
||||
# make one more attempt at figuring out the IP version
|
||||
if ((defined($sFlowSample->{'GatewayIpVersionNextHopRouter'}) &&
|
||||
$sFlowSample->{'GatewayIpVersionNextHopRouter'} == 2) ||
|
||||
(defined($sFlowSample->{'HeaderType'}) && $sFlowSample->{'HeaderType'} eq '86dd')) {
|
||||
$ipversion = 6;
|
||||
}
|
||||
}
|
||||
|
||||
my $srcas = 0;
|
||||
my $dstas = 0;
|
||||
|
||||
if ($sFlowSample->{'GatewayAsSource'}) {
|
||||
$srcas = $sFlowSample->{'GatewayAsSource'};
|
||||
}
|
||||
if ($sFlowSample->{'GatewayDestAsPaths'}) {
|
||||
$dstas = pop(@{$sFlowSample->{'GatewayDestAsPaths'}->[0]->{'AsPath'}});
|
||||
if (!$dstas) {
|
||||
$dstas = 0;
|
||||
}
|
||||
}
|
||||
|
||||
# Outbound packets have our AS number as the source (GatewayAsSource),
|
||||
# while inbound packets have 0 as the destination (empty AsPath).
|
||||
# Transit packets have "foreign" AS numbers for both source and
|
||||
# destination (handleflow() currently deals with those by counting
|
||||
# them twice; once for input and once for output)
|
||||
|
||||
# substitute 0 for own AS number
|
||||
if ($srcas == $myas) {
|
||||
$srcas = 0;
|
||||
}
|
||||
if ($dstas == $myas) {
|
||||
$dstas = 0;
|
||||
}
|
||||
|
||||
handleflow($ipaddr, $noctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion);
|
||||
}
|
||||
}
|
||||
|
||||
sub handleflow {
|
||||
my ($routerip, $noctets, $srcas, $dstas, $snmpin, $snmpout, $ipversion) = @_;
|
||||
|
||||
if ($srcas == 0 && $dstas == 0) {
|
||||
# don't care about internal traffic
|
||||
return;
|
||||
}
|
||||
|
||||
#print "$srcas => $dstas ($noctets octets, version $ipversion, snmpin $snmpin, snmpout $snmpout)\n";
|
||||
|
||||
# determine direction and interface alias name (if known)
|
||||
my $direction;
|
||||
my $ifalias;
|
||||
my $as;
|
||||
|
||||
if ($srcas == 0) {
|
||||
$as = $dstas;
|
||||
$direction = "out";
|
||||
$ifalias = $knownlinks{inet_ntoa($routerip) . '_' . $snmpout};
|
||||
} elsif ($dstas == 0) {
|
||||
$as = $srcas;
|
||||
$direction = "in";
|
||||
$ifalias = $knownlinks{inet_ntoa($routerip) . '_' . $snmpin};
|
||||
} else {
|
||||
handleflow($routerip, $noctets, $srcas, 0, $snmpin, $snmpout, $ipversion);
|
||||
handleflow($routerip, $noctets, 0, $dstas, $snmpin, $snmpout, $ipversion);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!$ifalias) {
|
||||
# ignore this, as it's through an interface we don't monitor
|
||||
return;
|
||||
}
|
||||
|
||||
my $dsname;
|
||||
if ($ipversion == 6) {
|
||||
$dsname = "${ifalias}_v6_${direction}";
|
||||
} else {
|
||||
$dsname = "${ifalias}_${direction}";
|
||||
}
|
||||
|
||||
# put it into the cache
|
||||
if (!$ascache->{$as}) {
|
||||
$ascache->{$as} = {createts => time};
|
||||
}
|
||||
|
||||
$ascache->{$as}->{$dsname} += $noctets;
|
||||
$ascache->{$as}->{updatets} = time;
|
||||
|
||||
if ($ascache->{$as}->{updatets} == $ascache_lastflush) {
|
||||
# cheat a bit here
|
||||
$ascache->{$as}->{updatets}++;
|
||||
}
|
||||
|
||||
# now flush the cache, if necessary
|
||||
flush_cache();
|
||||
}
|
||||
|
||||
sub flush_cache {
|
||||
|
||||
if ($childrunning || ((time - $ascache_lastflush) < $ascache_flush_interval)) {
|
||||
# can't/don't want to flush cache right now
|
||||
return;
|
||||
}
|
||||
|
||||
$childrunning = 1;
|
||||
my $pid = fork();
|
||||
|
||||
if (!defined $pid) {
|
||||
$childrunning = 0;
|
||||
print "cannot fork\n";
|
||||
} elsif ($pid != 0) {
|
||||
# in parent
|
||||
$ascache_lastflush = time;
|
||||
for (keys %$ascache) {
|
||||
if ($_ % 10 == $ascache_flush_number % 10) {
|
||||
delete $ascache->{$_};
|
||||
}
|
||||
}
|
||||
$ascache_flush_number++;
|
||||
return;
|
||||
}
|
||||
|
||||
while (my ($as, $cacheent) = each(%$ascache)) {
|
||||
if ($as % 10 == $ascache_flush_number % 10) {
|
||||
#print "$$: flushing data for AS $as ($cacheent->{updatets})\n";
|
||||
|
||||
my $rrdfile = getrrdfile($as, $cacheent->{updatets});
|
||||
my @templatearg;
|
||||
my @args;
|
||||
|
||||
while (my ($dsname, $value) = each(%$cacheent)) {
|
||||
next if ($dsname !~ /_(in|out)$/);
|
||||
|
||||
my $tag = $dsname;
|
||||
$tag =~ s/(_v6)?_(in|out)$//;
|
||||
my $cursamplingrate = $samplingrate;
|
||||
|
||||
if ($link_samplingrates{$tag}) {
|
||||
$cursamplingrate = $link_samplingrates{$tag};
|
||||
}
|
||||
|
||||
push(@templatearg, $dsname);
|
||||
push(@args, $value * $cursamplingrate);
|
||||
}
|
||||
|
||||
RRDs::update($rrdfile, "--template", join(':', @templatearg),
|
||||
$cacheent->{updatets} . ":" . join(':', @args));
|
||||
my $ERR = RRDs::error;
|
||||
if ($ERR) {
|
||||
print "Error updating RRD file $rrdfile: $ERR\n";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
exit 0;
|
||||
}
|
||||
|
||||
# create an RRD file for the given AS, if it doesn't exist already,
|
||||
# and return its file name
|
||||
sub getrrdfile {
|
||||
my $as = shift;
|
||||
my $startts = shift;
|
||||
$startts--;
|
||||
|
||||
# we create 256 directories and store RRD files based on the lower
|
||||
# 8 bytes of the AS number
|
||||
my $dirname = "$rrdpath/" . sprintf("%02x", $as % 256);
|
||||
if (! -d $dirname) {
|
||||
# need to create directory
|
||||
mkdir($dirname);
|
||||
}
|
||||
|
||||
my $rrdfile = "$dirname/$as.rrd";
|
||||
|
||||
# let's see if there's already an RRD file for this AS - if not, create one
|
||||
if (! -r $rrdfile) {
|
||||
#print "$$: creating RRD file for AS $as\n";
|
||||
|
||||
my %links = map { $_, 1 } values %knownlinks;
|
||||
|
||||
my @args;
|
||||
foreach my $alias (keys %links) {
|
||||
push(@args, "DS:${alias}_in:ABSOLUTE:300:U:U");
|
||||
push(@args, "DS:${alias}_out:ABSOLUTE:300:U:U");
|
||||
push(@args, "DS:${alias}_v6_in:ABSOLUTE:300:U:U");
|
||||
push(@args, "DS:${alias}_v6_out:ABSOLUTE:300:U:U");
|
||||
}
|
||||
push(@args, "RRA:AVERAGE:0.99999:1:576"); # 48 hours at 5 minute resolution
|
||||
push(@args, "RRA:AVERAGE:0.99999:12:168"); # 1 week at 1 hour resolution
|
||||
push(@args, "RRA:AVERAGE:0.99999:48:180"); # 1 month at 4 hour resolution
|
||||
push(@args, "RRA:AVERAGE:0.99999:288:366"); # 1 year at 1 day resolution
|
||||
RRDs::create($rrdfile, "--start", $startts, @args);
|
||||
|
||||
my $ERR = RRDs::error;
|
||||
if ($ERR) {
|
||||
print "Error creating RRD file $rrdfile: $ERR\n";
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
return $rrdfile;
|
||||
}
|
||||
|
||||
sub read_knownlinks {
|
||||
my %knownlinks_tmp;
|
||||
my %link_samplingrates_tmp;
|
||||
open(KLFILE, $knownlinksfile) or die("Cannot open $knownlinksfile!");
|
||||
while (<KLFILE>) {
|
||||
chomp;
|
||||
next if (/(^\s*#)|(^\s*$)/); # empty line or comment
|
||||
|
||||
my ($routerip,$ifindex,$tag,$descr,$color,$samplingrate) = split(/\t+/);
|
||||
$knownlinks_tmp{"${routerip}_${ifindex}"} = $tag;
|
||||
|
||||
if ($samplingrate) {
|
||||
$link_samplingrates_tmp{$tag} = $samplingrate;
|
||||
}
|
||||
}
|
||||
close(KLFILE);
|
||||
|
||||
%knownlinks = %knownlinks_tmp;
|
||||
%link_samplingrates = %link_samplingrates_tmp;
|
||||
return;
|
||||
}
|
||||
@ -1,10 +1,10 @@
|
||||
# Router IP SNMP ifindex tag description color samplingrate (optional)
|
||||
# Router IP SNMP ifindex tag description color samplingrate
|
||||
# note: tabs must be used to separate fields (not spaces)
|
||||
# max. length for tag is 12 characters
|
||||
192.0.2.1 15 uplink1 Uplink 1 A6CEE3
|
||||
192.0.2.1 23 uplink2 Uplink 2 1F78B4
|
||||
192.0.2.2 4 uplink3 Uplink 3 B2DF8A
|
||||
192.0.2.3 11 uplink4 Uplink 4 33A02C
|
||||
192.0.2.2 42 peering1 IXP 1 FB9A99
|
||||
192.0.2.2 45 peering2 IXP 2 E31A1C
|
||||
192.0.2.3 6 peering3 IXP 3 FDBF6F
|
||||
192.0.2.1 15 uplink1 Uplink 1 A6CEE3 1
|
||||
192.0.2.1 23 uplink2 Uplink 2 1F78B4 1
|
||||
192.0.2.2 4 uplink3 Uplink 3 B2DF8A 1
|
||||
192.0.2.3 11 uplink4 Uplink 4 33A02C 1
|
||||
192.0.2.2 42 peering1 IXP 1 FB9A99 1
|
||||
192.0.2.2 45 peering2 IXP 2 E31A1C 1
|
||||
192.0.2.3 6 peering3 IXP 3 FDBF6F 2048
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
<div id="footer">
|
||||
AS-Stats v1.43 written by Manuel Kasper for Monzoon Networks AG.<br/>
|
||||
AS-Stats v1.5b written by Manuel Kasper for Monzoon Networks AG.<br/>
|
||||
<?php if ($outispositive): ?>
|
||||
Outbound traffic: positive / Inbound traffic: negative
|
||||
<?php else: ?>
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user