malhuda/AS-Stats
1
0
Fork 0
mirror of https://github.com/manuelkasper/AS-Stats.git synced 2025-02-20 11:44:12 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
AS-Stats/README

225 lines
8.4 KiB
Plaintext
Raw Blame History

AS-Stats v1.32 (2010-08-02)
a simple tool to generate per-AS traffic graphs from NetFlow/sFlow records
by Manuel Kasper, Monzoon Networks AG <mkasper@monzoon.net>
--------------------------------------------------------------------
Changes
-------
xxx Fix for multiple entries with the same tag in the
knownlinks file (e.g. for LACP)
(contributed by Michal Buchtik)
v1.32 Fix add_ds.sh to support new directory structure
(contributed by Sergei Veltistov)
Fix PHP warnings and move $ntop to config.inc
(contributed by Michal Buchtik)
v1.31 Set memory_limit to 256 MB in PHP pages (suggested by Steve Colam
<steve@colam.co.uk>).
Allow NetFlow aggregation version 0 as well as 2 to make NetFlow
exports from Juniper routers work (suggested by Thomas Mangin
<thomas.mangin@exa-networks.co.uk>).
Updated asinfo.txt with data from whois.cymru.com.
v1.3 Changes by Steve Colam <steve@colam.co.uk>:
- ...-asstatd.pl now accepts parameters (UDP listen port,
sampling rate etc.) on the command line
****************************************************
* Mind the new command line syntax when upgrading! *
* Change your start script accordingly... *
****************************************************
- hierarchical RRD structure for more efficient storage
(one directory per low byte of AS number)
*********************************************
* Use tools/migraterrdfiles.pl to move your *
* RRD files when upgrading! *
*********************************************
- ...-asstatd.pl now re-reads known links file upon SIGHUP
Added contrib/generate-asinfo.py script to generate AS list from WHOIS
data (contributed by Thomas Mangin
<thomas.mangin@exa-networks.co.uk>).
Moved site-specific parameters of www frontend to config.inc.
New flag images from famfamfam.com.
Updated asinfo.txt.
v1.2 Support for sFlow (through sflow-asstatd.pl); fix for link names
with upper-case characters
Allow inbound/outbound in graphs to be swapped (via option
in www/config.inc)
v1.1 Fix for a potential race condition surrounding $childrunning
(reported by Yann Gauteron; experienced on a Linux system)
v1 Initial release
How it works
------------
A Perl script (netflow-asstatd.pl) collects NetFlow v8 AS aggregation records
or sFlow v5 samples from one or more routers. It caches them for about a
minute (to prevent excessive writes to RRD files), identifies the link that
each record refers to (by means of the SNMP in/out interface index), maps it
to a corresponding "known link" and RRD data source, and then runs RRDtool. To
avoid losing new records while the RRD files are updated, the update task is
run in a separate process.
For each AS, a separate RRD file is created as needed. It contains two data
sources for each link - one for inbound and one for outbound traffic.
In generated per-AS traffic graphs, inbound traffic is shown as positive,
while outbound traffic is shown as negative values.
Another Perl script, rrd-extractstats.pl, is meant to run about once per hour.
It sums up per-AS and link traffic during the last 24 hours, sorts the ASes
by total traffic (descending) and writes the results to a text file. This
is then used to display the "top N AS" and other stats by the provided PHP
scripts.
Prerequisites
-------------
- Perl 5.8
- RRDtool 1.2 (with Perl "RRDs" library)
- web server with PHP 5
- one or more routers than can generate NetFlow v8 AS aggregation records
or sFlow samples
Installation
------------
In the instructions below, "xx-asstatd.pl" refers to either netflow-asstatd.pl
or sflow-asstatd.pl, depending on whether your routers generate NetFlow or
sFlow data.
- Copy the perl scripts xx-asstatd.pl and rrd-extractstats.pl to the
machine that will collect NetFlow/sFlow records
- Create a "known links" file with the following information about each
link that you want to appear in your AS stats:
- IP address of router (= source IP of NetFlow datagrams)
- SNMP interface index of interface (use "show snmp mib ifmib ifindex"
to find out)
- a short "tag" (15 chars max., alphanumerics only) that will be used
internally (e.g. for RRD DS names)
- a human-readable description (will appear in the generated graphs)
- a color code for the graphs (HTML style, 6 hex digits)
See the example file provided (knownlinks) for the format.
- Create a directory to hold per-AS RRD files. For each AS, about 128 KB of
storage are required, and there could be (in theory) up to 64511 ASes.
AS-Stats automatically creates 256 subdirectories in this directory for
more efficient storage of RRD files (one directory per lower byte of
AS number, in hex).
- Start xx-asstatd.pl in the background (or, better yet, write a
startup script for your operating system to automatically start
xx-asstatd.pl on boot):
nohup xx-asstatd.pl -r /path/to/rrd/dir -k /path/to/knownlinks &
By default, netflow-asstatd.pl will listen on port 9000 (UDP) for NetFlow
datagrams, and sflow-asstatd.pl will listen on port 6343 (UDP) for sFlow
datagrams. Use the -p option if you want to change that.
If you use sampled NetFlow or sFlow, set the sampling rate with the -s
option.
sflow-asstatd.pl also needs you to specify your own AS number with the -a
option for accurate classification of inbound and outbound traffic.
It's a good idea to make sure only UDP datagrams from your trusted routers
will reach the machine running xx-asstatd.pl (firewall etc.).
- NetFlow only:
Have your router(s) send NetFlow v8 AS aggregation records to your machine.
This is typically done with commands like the following (Cisco IOS):
ip flow-cache timeout active 5
int Gi0/x.y
ip flow ingress
ip flow-export source <source interface>
ip flow-export version 5 origin-as
ip flow-aggregation cache as
cache timeout active 5
export destination <IP address of server running AS stats> 9000
enabled
Note that the version has to be specified as 5, even though the AS
aggregation records will actually be v8. Also, setting the global flow
cache timeout to 5 minutes is necessary to get "smooth" traffic graphs
(default is 30 minutes), as a flow is only counted when it expires from
the cache. Decreasing the flow-cache timeout may result in a slight
increase in CPU usage (and NetFlow AS aggregation takes its fair share of
CPU as well, of course).
- sFlow only:
Have your router(s) send sFlow samples to your machine. Your routers
may need a software upgrade to make them include AS path information for
both inbound and outbound packets (this is a good thing to check if
your graphs only show traffic on one direction).
- Wait 1-2 minutes. You should then see new RRD files popping up in the
directory that you defined/created earlier on. If not, make sure that
xx-asstatd.pl is running, not spewing out any error messages, and that
the NetFlow/sFlow datagrams are actually reaching your machine (tcpdump...).
- Add a cronjob to run the following command every hour:
rrd-extractstats.pl /path/to/rrd/dir /path/to/knownlinks \
/path/to/asstats_day.txt
That script will go through all RRD files and collect per-link summary
stats for each AS, sort them by total traffic (descending), and write them
to a text file. The "top N AS" page uses this to determine which ASes to
show.
- Copy the contents of the "www" directory to somewhere within your web
server's document root and change file paths in config.inc as necessary.
- Wait a few hours for data to accumulate. :)
- Access the provided PHP scripts via your web server and marvel at the
(hopefully) beautiful graphs.
Adding a new link
-----------------
Adding a new link involves adding two new data sources to all RRD files.
This is a bit of a PITA since RRDtool itself doesn't provide a command to do
that. A simple (but slow) Perl script that is meant to be used with RRDtool's
XML dump/restore feature is provided (add_ds_proc.pl, add_ds.sh). Note that
netflow-asstatd.pl should be stopped while modifying RRD files, to avoid
breaking them with concurrent modifications.
Changing the RRAs
-----------------
By default, the created RRDs keep data as follows:
* 48 hours at 5 minute resolution
* 1 week at 1 hour resolution
* 1 year at 1 day resolution
If you want to change that, modify the getrrdfile() function in
xx-asstatd.pl and delete any old RRD files.
To do
-----
- rrd-extractstats.pl uses a lot of memory and could probably use some
optimization.
Reference in New Issue View Git Blame Copy Permalink