Welcome To suyashjain.blogspot.com

For Latest and more contents visit http://www.i3w.in

Monday, December 22, 2008

Cacti & iptables

#!/usr/bin/perl # # This is a quick perl script to  # pull bandwidth usage from iptables chains # # If you use/optimize this script, please let me know. # Brian Stanback : brian [at] stanback [dot] net  # Example iptables rule for web bandwidth usage: # > iptables -N WWW # > iptables -A WWW -j ACCEPT # > iptables -A INPUT -p tcp -m tcp --dport 80 -j WWW # > iptables -A OUTPUT -p tcp -m tcp --sport 80 -j WWW # # Run "iptables.pl WWW" as root to test, note that you can  # combine more than one protocol into a single chain. # # Sudo Configuration (/etc/sudoers) # > www-data    ALL = NOPASSWD: /usr/share/cacti/scripts/iptables.pl # # The Input String should be set to "sudo <path_cacti>/scripts/iptables.pl <chain>" # and you will need to setup an input field so that the <chain> argument can be passwd. # # The data input type should be set to COUNTER #  if ($ARGV[0]) {         $chains = `/sbin/iptables -xnvL | grep -A 2 'Chain $ARGV[0]'`;         @chains = split(/\n/, $chains);         $chains[2] =~ /[\W+]?[0-9]+\W+([0-9]+)\W+/;         print $1; } else {         print "Usage: $0 Chain\n"; } 
 
http://rodotelmi.rebstech.com/2008/06/30/cacti-with-iptablesipfw-traffic-monitoring/
 
 

Monday, December 15, 2008

Ossec - What Exactly it is

Ossec - Open Source Host Intrusion Detection System

Thursday, December 11, 2008

Hiding the apache Identity

To hide the version and other information of apache server which can be
retrieved through header of an request,
put the following lines in your apache httpd.conf file.


RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^TRACE
RewriteRule .* - [F]
ServerSignature off
ServerTokens Prod

Through these lines you are hiding the signature of apache and the
header will only display 'Apache'.

How to Block Torrent in Network

Torrent works based on the seeds and peers (the other computer which connects to your client application to send or receive the files) .

The seeds and peers  information to your client is provided by the torrent tracker server, which is mentioned in the  .torrent file downloaded by you from some torrent site.

Your Torrent client will read the .torrent file and connect with the torrent tracker server on 6881-7000/tcp or 2710/tcp port through http protocol and torrent tracker server will provide all the other client computer which are currently connected and having the full file or partial file which yuo want to download.

Than your client application(torrent client) will connect to those other client machines(torrent client applications) and starting transferring the data.

To block the torrent in your network , you can take the following two actions.

1. Do not allow "NEW" incoming packets in your network.

2. Block 6881-7000/tcp and 2710/tcp ports for outgoing so that your client application(torrent client) could not  connect the tracker server and get the information about other systems.

If your client(torrent application) will not get the information about other systems in world , it will not be able to download the files.

This is want you want to do.....

Cheer Up !!

Your comments and experience are most welcome.

Friday, October 17, 2008

Cacti Password Hacking

I have found one password hacking trick which can be used for cacti to
change any user password including admin.

http://xxx.xxx.xxx.xxx/auth_changepassword.php?ref=index.php&action=changepassword&username=admin&password=aaaaaa&confirm=aaaaaa&submit=Save

xxx.xxx.xxx.xxx - ip of cacti server.


This url will provide option to change the password of admin user. if
the same is happening with your cacti also kindly block
auth_changepassword.php file from accessing through web.

Friday, October 10, 2008

BIOS Blaster and Capture Utilities

 11th Alliance toolkit  =>    Another toolkit containing utilities needed to gain access to a wide range  of BIOS passwords. Download it from  http://www.wheres.com/etc/FatherQuinn/bios310.zip.
 AMIDECOD    =>        This utility will decode BIOS passwords on American Megatrends systems. Get it at http://www.outpost9.com/files/crackers.html.
 AMI Password Viewer   =>      This utility from KORT reads, decrypts, and displays AMI BIOS passwords. Get it at http://www.rat.pp.se/hotel/panik/archive/skw-ami.zip.
  AW.COM     =>   This utility by Falcon n Alex cracks Award BIOS passwords. Get it at http://www.lls.se/~oscar/files/pwd/aw.zip.
  CmosPwd     =>  CmosPwd can retrieve BIOS passwords from many popular computers,including IBM, Compaq, Packard Bell, and Gateway. Download it at   http://www.esiea.fr/public_html/Christophe.GRENIER/index.html?cmospwd.html.
  Kill CMOS   => If a user-defined password already exists on a computer, resetting the CMOS to its default state will erase that password. A utility to do this can be downloaded from http://www.AntiOnline.com/archives/anticode/bios-crackers/killcmos.zip.

Wednesday, October 01, 2008

Ethical Hacking - Footprinting - SmartWhois

SmartWhois is a windows based GUI version of Whois. It is an
information-gathering program that allows you to find all available
information about an IP address, host name, or domain, including
country, state or province, city,name of the network provider,
administrator, and technical-support contact information.

You can get it from here.

http://www.tamos.com/products/smartwhois/

Ethical Hacking - FootPrinting - Sam Spade

Sam Spade is windows based GUI tool which provide all in one interface for Foot Printing. It includes Ping, DNS, Whois, IP Block info, Dig, Traceroute etc

· Each tool displays it's output in it's own window, and everything is multi-threaded so you don't need to wait for one query to complete before starting the next one
· Some functions are threaded still further to allow lazy reverse DNS lookups (never do a traceroute -n again)
· The output from each query is hotlinked, so you can right click on an email address, IP address, hostname or internic tag to run another query on it
· Appending the results of a query to the log window is a single button function
· There's a lot of online help, in both WinHelp and HTMLHelp formats. This includes tutorials, background information and links to online resources as well as the program manual itself


You can download it from the following location.

http://www.softpedia.com/get/Network-Tools/Network-Tools-Suites/Sam-Spade.shtml

Ethical Hacking - Foorprinting - Traceroute

traceroute is a command which is available in all OS.

If you the ip domain/ip address of organization , you can trace the gateway devices and ISP details.

Example:

Tracing route to yahoo.com [68.180.206.184]...

hop rtt rtt rtt   ip address domain name
1 8 1 0   70.84.211.97 61.d3.5446.static.theplanet.com
2 0 0 0   70.84.160.162 vl2.dsr02.dllstx5.theplanet.com
3 0 0 0   70.85.127.109 po52.dsr02.dllstx3.theplanet.com
4 0 0 0   70.87.253.21 et3-1.ibr03.dllstx3.theplanet.com
5 0 0 0   70.87.253.178 b2.fd.5746.static.theplanet.com
6 23 21 22   216.115.96.58 so-4-0-0.pat2.dnx.yahoo.com
7 49 49 63   216.115.101.128 as0.pat1.pao.yahoo.com
8 48 47 47   216.115.101.33 ae2.pat2.pao.yahoo.com
9 49 48 48   216.115.107.51 ae0-p141.msr1.sp1.yahoo.com
10 49 48 49   209.131.32.23 te-9-1.bas-a1.sp1.yahoo.com
11 47 47 48   68.180.206.184 w2.rc.vip.sp1.yahoo.com

Trace complete.

Traceroute.org is one of the fantastic site from where you can perform the traceroute through various part of world.

Ethical Hacking - FootPrinting - DIG

Dig is a linux command which is similor to nslookup command. It also comes under DNS enumeration.

Example:

dig yahoo.com

; <<>> DiG 9.3.3rc2 <<>> yahoo.com

;; global options: printcmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39856

;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 7, ADDITIONAL: 7


;; QUESTION SECTION:

;yahoo.com. IN A


;; ANSWER SECTION:

yahoo.com. 359 IN A 68.180.206.184

yahoo.com. 359 IN A 206.190.60.37


;; AUTHORITY SECTION:

yahoo.com. 52791 IN NS ns4.yahoo.com.

yahoo.com. 52791 IN NS ns5.yahoo.com.

yahoo.com. 52791 IN NS ns6.yahoo.com.

yahoo.com. 52791 IN NS ns8.yahoo.com.

yahoo.com. 52791 IN NS ns1.yahoo.com.

yahoo.com. 52791 IN NS ns2.yahoo.com.

yahoo.com. 52791 IN NS ns3.yahoo.com.


;; ADDITIONAL SECTION:

ns1.yahoo.com. 52959 IN A 66.218.71.63

ns2.yahoo.com. 52959 IN A 68.142.255.16

ns3.yahoo.com. 52959 IN A 217.12.4.104

ns4.yahoo.com. 52959 IN A 68.142.196.63

ns5.yahoo.com. 66635 IN A 119.160.247.124

ns6.yahoo.com. 17127 IN A 202.43.223.170

ns8.yahoo.com. 52790 IN A 202.165.104.22


;; Query time: 2 msec

;; SERVER: 202.71.152.65#53(202.71.152.65)

;; WHEN: Tue Sep 30 19:38:35 2008

;; MSG SIZE rcvd: 297


Different Types of DNS Records

The following list describes the common DNS record types and their use:

A (address)—Maps a host name to an IP address

SOA (Start of Authority)—Identifies the DNS server responsible for the domain information

CNAME (canonical name)—Provides additional names or aliases for the address record

MX (mail exchange)—Identifies the mail server for the domain

SRV (service)—Identifies services such as directory services

PTR (pointer)—Maps IP addresses to host names

NS (name server)—Identifies other name servers for the domain

Ethical Hacking - Footprinting - NSLOOKUP

nslookup is one of the fantastis tool through which dns enumeration can be performed.

It can provides the following informations.


  1. Ip addresses

  2. Domain names

  3. Sub domain names or computer names

  4. Mail Servers

  5. DNS Server


Sample Output

# nslookup

> set type=mx

> google.com

Server: x.x.x.x

Address: x.x.x.x#53


Non-authoritative answer:

google.com mail exchanger = 10 smtp3.google.com.

google.com mail exchanger = 10 smtp4.google.com.

google.com mail exchanger = 10 smtp1.google.com.

google.com mail exchanger = 10 smtp2.google.com.


Authoritative answers can be found from:

google.com nameserver = ns1.google.com.

google.com nameserver = ns2.google.com.

google.com nameserver = ns3.google.com.

google.com nameserver = ns4.google.com.

ns1.google.com internet address = 216.239.32.10

ns2.google.com internet address = 216.239.34.10

ns3.google.com internet address = 216.239.36.10

ns4.google.com internet address = 216.239.38.10

> exit


online nslookup is also available.

networking.ringofsaturn.com/Tools/nslookup.php

centralops.net/
www.nexperts.org/onlinenslookup.aspx
www.subnetonline.com/pages/network-tools/online-nslookup.php
enc.com.au/itools/nslookup.php

Ethical Hacking - FootPrinting - Host

Linux host command can also be used to provide a lot of information about domain.

Example:

host yahoo.com

yahoo.com has address 68.180.206.184

yahoo.com has address 206.190.60.37

yahoo.com mail is handled by 1 c.mx.mail.yahoo.com.

yahoo.com mail is handled by 1 d.mx.mail.yahoo.com.

yahoo.com mail is handled by 1 e.mx.mail.yahoo.com.

yahoo.com mail is handled by 1 f.mx.mail.yahoo.com.

yahoo.com mail is handled by 1 g.mx.mail.yahoo.com.

yahoo.com mail is handled by 1 a.mx.mail.yahoo.com.

yahoo.com mail is handled by 1 b.mx.mail.yahoo.com.

Ethical Hacking - FootPrinting - WHOIS

Whois searches the Internet for domain name administration details, such as domain ownership, address, location, phone number, dns servers and so on, about a specific domain name.

Whois tool is available in all linux OS as well as other OS also. Whois is also availabe online.

Example:

whois eccouncil.org

[Querying whois.publicinterestregistry.net]

[whois.publicinterestregistry.net]

NOTICE: Access to .ORG WHOIS information is provided to assist persons in

determining the contents of a domain name registration record in the Public Interest Registry

registry database. The data in this record is provided by Public Interest Registry

for informational purposes only, and Public Interest Registry does not guarantee its

accuracy. This service is intended only for query-based access. You agree

that you will use this data only for lawful purposes and that, under no

circumstances will you use this data to: (a) allow, enable, or otherwise

support the transmission by e-mail, telephone, or facsimile of mass

unsolicited, commercial advertising or solicitations to entities other than

the data recipient's own existing customers; or (b) enable high volume,

automated, electronic processes that send queries or data to the systems of

Registry Operator or any ICANN-Accredited Registrar, except as reasonably

necessary to register domain names or modify existing registrations. All

rights reserved. Public Interest Registry reserves the right to modify these terms at any

time. By submitting this query, you agree to abide by this policy.


Domain ID:D81180127-LROR

Domain Name:ECCOUNCIL.ORG

Created On:14-Dec-2001 10:13:06 UTC

Last Updated On:22-Jul-2008 13:44:54 UTC

Expiration Date:14-Dec-2014 10:13:06 UTC

Sponsoring Registrar:Tucows Inc. (R11-LROR)

Status:OK

Registrant ID:tudWNRx8nZtFHrvG

Registrant Name:Technical Support

Registrant Organization:EC-Council

Registrant Street1:3819 Osuna Rd NE

Registrant Street2:

Registrant Street3:

Registrant City:Albuquerque

Registrant State/Province:NM

Registrant Postal Code:87109

Registrant Country:US

Registrant Phone:+1.2127098253

Registrant Phone Ext.:

Registrant FAX:

Registrant FAX Ext.:

Registrant Email:info@eccouncil.org

Admin ID:tu5CH8cTqPzxTAEi

Admin Name:Technical Support

Admin Organization:EC-Council

Admin Street1:3819 Osuna Rd NE

Admin Street2:

Admin Street3:

Admin City:Albuquerque

Admin State/Province:NM

Admin Postal Code:87109

Admin Country:US

Admin Phone:+1.2127098253

Admin Phone Ext.:

Admin FAX:

Admin FAX Ext.:

Admin Email:info@eccouncil.org

Tech ID:tu8jivUXxCudWa9J

Tech Name:Technical Support

Tech Organization:EC-Council

Tech Street1:3819 Osuna Rd NE

Tech Street2:

Tech Street3:

Tech City:Albuquerque

Tech State/Province:NM

Tech Postal Code:87109

Tech Country:US

Tech Phone:+1.2127098253

Tech Phone Ext.:

Tech FAX:

Tech FAX Ext.:

Tech Email:info@eccouncil.org

Name Server:AUTH1.NS.NYI.NET

Name Server:AUTH2.NS.NYI.NET

Name Server:

Name Server:

Name Server:

Name Server:

Name Server:

Name Server:

Name Server:

Name Server:

Name Server:

Name Server:

Name Server:


Online Whois Query Sites:
networking.ringofsaturn.com/Tools/whois.php
www.arin.net/whois/
whois.domaintools.com/
www.onlinewhois.org/
www.subnetonline.com/pages/network-tools/online-whois.php
www.whoistheowner.net/
samspade.org/
network-tools.com/
http://centralops.net/co/


Tuesday, September 30, 2008

Google Hacking - intitle: "BorderManager information alert"

to look for Novell BorderManager Proxy/Firewall servers.

Google Hacking Database

http://johnny.ihackstuff.com/ghdb.php

Google Hacking

Query : intitle:"Welcome to IIS 4.0"
A listing of Windows IIS 4.0 servers, which have had a plethora of security vulnerabilities, and are usually easy pickings for most attackers.

Query: "VNC Desktop" inurl:5800
VNC Server allows remote users to connect and control a user's desktop. It is possible for this service to be configured without a password and allow direct access to the desktop.

Query: filetype:pwd service

A quick click on one of the links reveals several usernames and UNIX passwords:
# -FrontPage-
ekendall:bYld1Sr73NLKo
louisa:5zm94d7cdDFiQ

Query: filetype:bak inurl:"htaccess|passwd|shadow|htusers"

reveals all kinds of information related to password files that store usernames and encrypted passwords (which can easily be cracked).

Query : filetype:properties inurl:db intext:password

A quick click on one of the results reveals database passwords in clear text!drivers=sun.jdbc.odbc.JdbcOdbcDriver jdbc.idbDriver
logfile=D:\\user\\src\\java\\DBConnectionManager\\log.txt
idb.url=jdbc:idb:c:\\local\\javawebserver1.1\\db\\db.prp
 
 
idb.maxconn=2
 
 
access.url=jdbc:odbc:demo
access.user=demo
access.password=demopw

Query: "not for distribution" confidential site:edu

Over 100 confidential documents are revealed at the click of a button. Too bad that university left their students' social security numbers in that PDF document.

Query: This file was generated by Nessus

Nessus is a very popular vulnerability scanner that many administrators use. Unfortunately for the unsuspecting victims, Joe Hacker has now located hundreds of Nessus reports that have inadvertently been left on users' systems. This is an amazing bounty of systems accessible via the Internet that provides a blueprint of all their vulnerabilities!.

Note : This information is taken from Hacking Exposed 5th Edition.

Tuesday, August 19, 2008

Password input from command line in shell script

Some time you are writing a shell script which asks for some secret
answer which you don't want to display on screen. To avoid echoing the
input entered by users , do this.

setty -echo
read secretanswer
setty echo

First line will disable the echo
Second line will take the input , but will not display whatever user typed
Third line again enable the echo

Thursday, July 31, 2008

Dynagen and Dynamips , Cisco Router Simulator

Dynagen and Dynamips are frontend and backend application for Cisco router simulator. It uses the actual ios image files of cisco , so you will be able to user all commands which are  available in that ios version.


For starting you require three things.

1. Dynamips executable file  = > something like dynamips-0.2.8-RC2-x86.bin

http://www.ipflow.utc.fr/dynamips/dynamips-0.2.7-x86.bin

2. Dynagen => download the dynagen tar.gz
3. Cisco IOS image - At this time, dynamips is able to boot a large number of Cisco IOS releases available for the 7200, 3600, 3700 and 2600 platforms, including the latest 12.2S and 12.4T.



Lets Start Now

First create a network lab file like this. and save as network.net

##########################################################
[localhost]

    [[7200]]
    image =/Yash/c7200-is-mz.123-6b.bin
    npe = npe-400
    ram = 160

    [[router R2]]
    s1/1 = R3 s1/1

    [[router R3]]
    s1/2 = R4 s1/2

    [[router R4]]
    s1/1 = R2 s1/2

    [[router R5]]
    s1/3 = R2 s1/3
    # No need to specify an adapter here, it is taken care of
    # by the interface specification under Router R1

###############################################################
In this file

first line says that server has to run on localhost
second line says the cisco router version for which we will run it.
third line is the specification of image file.
4/5 i don;t know

from 6 i have created the network topology.


now run

./dynamips-0.2.8-RC2-x86.bin 7200 -H c7200-is-mz.123-6b.bin

this will run dynamips server.

than run

./dynagen network.net ,

this will give you the access of IOS image

Wednesday, July 30, 2008

Syslog Logger in Perl

#!/usr/bin/perl
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Library General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
#################################################################
# Name : syslog-fifo.pl
# Summary : Syslog fifo script
# Author : Tony Green
# Date : Dec '00
# Synopsis : Syslog automation script
#################################################################
#
# Hash array to seperate messages into individual files
# To add a new entry use the following syntax
# "5th Column Title" => ".suffix of file"
# You will need to restart the script.
%suffix = (
"sendmail" => ".mail",
"sshd" => ".ssh",
"su" => ".auth",
"PAM_pwdb" => ".auth",
"cron" => ".cron",
"crond" => ".cron",
"vpop3d" => ".pop",
"named" => ".named",
"named-xfer" => ".named",
"nscd" => ".named",
"ftp" => ".ftp",
"ftpd" => ".ftp",
"generic" => ".messages",
"capacity" => ".capacity",
"housekeeping" => ".housekeeping",
"prtdiag" => ".prtdiag",
"savecore" => ".crash",
"snmpdx" => ".snmp",
"perfomance" => ".perf",
"disaster-recovery" => ".disaster",
"httpd" => ".apache"
);


%alerts = (
"portwatch" => 'suyash.j@net4.in',
"file system full" => 'nocbang@net4india.net',
);

my $id = 'syslog-fifo.pl';
my $pidfile = '/var/run/syslog-fifo.pid';
my $warn = 'suyash.j\@net4.in';
my $basedir = '/var/log';
my $fifo = "$basedir/messages.fifo";
my $bit = "$basedir/unknown.messages";


# Check if we are already running
if ( -f $pidfile )
{
open (PID,$pidfile) or die "Couldn't open $pidfile\n";
$oldpid = <PID>;
close (PID) or die "Couldn't close $pidfile\n";
open (PS,"ps -fp $oldpid|") or die "Couldn't open ps stream\n";
while (<PS>)
{
next if m/UID/;
die "$id seems to be running already (pid $oldpid)" if m/$id/;
}
close (PS);
unlink $pidfile;
}

open (PID,">$pidfile") or die "Couldn't write to $pidfile\n";
print PID $$;
close (PID) or die "Couldn't close $pidfile\n";


%months = (
"Jan" => "1",
"Feb" => "2",
"Mar" => "3",
"Apr" => "4",
"May" => "5",
"Jun" => "6",
"Jul" => "7",
"Aug" => "8",
"Sep" => "9",
"Oct" => "10",
"Nov" => "11",
"Dec" => "12"
);


if ( -e $fifo && -r $fifo )
{ die "$fifo is not a pipe\n" unless -p $fifo }
else
{ die "$fifo is not there\n" }


# Open the 'input stream' from the named pipe
open (FIFO, $fifo) || die "Couldn't open fifo file";


# Turn into a 'daemon'
while ( 1 == 1 )
{
# While we have lines in the named pipe do this loop
while ( <FIFO> )
{
next if m/last message repeated/;
next if m/pam_setcred/;
next if m/Cannot delete credentials/;

&checkline($_);

chomp ($_);

@line = split (/\ +/, $_);

# Try to ensure there is a valid line by matching the 'tag'
#next unless ( $line[4] =~ m/\w*:$/ && $line[4] =~ m/last/ );

# Specifiy the month field
$month = "$line[0]";

$monthtest = $months{$month};

if ( ! defined($monthtest))
{
# This isn't a valid line - dump
# it into the 'bit bucket'

open (BIT, $bit);
print BIT "$_\n";
close (BIT);
}
else
{

# Specifiy the host field
$host = "$line[3]";
$host =~ tr/[A-Z]/[a-z]/;
my $domain = $host;
$host =~ s/\..+//;
$domain =~ s/$host\.//;
if ($domain =~ m/$host/)
{ $domain = 'net4india.net'; }

# Specifiy the type field
$type = "$line[4]";

# Get rid of the colon from the type field
$type =~ s/://;

# Get rid of the pid from the type field
$type =~ s/\[.*\]//;

# Build up the directory for storing the log files
$dir = $basedir . "/" . $month;

# Create the logdir/month directory if its not already there
&createlogdir($dir);

# Build up the directory for storing the log files
$dir = $dir . "/" . $domain;

# Create the logdir/month directory if its not already there
&createlogdir($dir);

# Add the host onto the logdir variable
$dir = $dir . "/" . $host;

# Create the logdir/month/host directory if this not
already there
&createlogdir($dir);

# From the type variable, figure out if this line should
be split
# into a different file by trying to access the 'type'
# in the %suffix hash array
$filetype = $suffix{$type};

# If $filetype is not defined it means that it did not
find its
# 'type' in the hash array, therefore we want it in the
bit bucket
if ( ! defined($filetype))
{
$filetype = $suffix{generic};
}


# Build up the 'output' stream
$file = ">> " . $dir . "/" . $host . $filetype;
$basefile = $dir . "/" . $host . $filetype;

# Open the output stream, write the line and close it again.
#&checkfilesize($basefile);
open (OUTPUT, $file) || die "Could not open $file";
print OUTPUT "$_\n";
close (OUTPUT);
}
# Slow things down just a touch to ensure we get everything
# through from the named pipe before parsing it.
select(undef, undef, undef, 0.05);
}

# Since we got an EOF from the fifo, give it a few seconds and
# then try again.
# Only new lines will be written into the named pipe so we can
# happily just process what has been written into it
sleep 5;
}

# Close the fifo at the end of the script
close ( FIFO );


###############################################################################
# Sub routines
###############################################################################


# Create subdirectories
sub createlogdir
{
if ( ! -d $_[0] )
{
mkdir ($_[0], 0755) || die "Could not create $_[0]\n";
print "Creating $_[0]\n";
}
}

# Simple sub to email out important messages
sub checkline
{
my $line = shift;
foreach $key (keys(%alerts))
{
if ( $line =~ m/$key/i )
{
open (MAIL, "|mailto -s 'Found $key in syslog' $alerts{$key}")
|| die "Couldn't open mail stream";
print MAIL $_;
close (MAIL);
}
}
}

sub checkfilesize
{
my $file = $_[0];
my $size = (stat($file))[7];
if ( $size > 10000 ) { system('gzip $file'); }
}

Friday, July 25, 2008

Netflow Tools

Pointer Collections

Cisco NetFlow Ecosystem Solutions
Contains descriptions of many applications that integrate NetFlow support, including a few "freeware" ones.
Cisco now lists Netflow applications on their Web pages, specifically applications from Cisco, commercial, and "freeware" applications.
Network Uptime list of Free NetFlow Tools
Nice overview of Netflow tools with screenshots.
FreshMeat NetFlow projects
A list of pointers to open-source projects related to NetFlow.

NetFlow

NFDUMP and NfSen
NFDUMP is a set of tools to capture/record, dump, filter, and replay NetFlow (v5/v7/9) data. Can filter flows according to multiple user-defined profiles. NfSen is a Graphical Web-based front-end for the NFDUMP tools. Plots aggregate statistics over time, supports filtering and drilling down up to the individual flow level.
CoMo
Traffic monitoring toolkit from Intel Research. Supports both continuous real-time processing and retrospective processing. Supports Netflow and many other traffic capture sources.
YAF - Yet Another Flow sensor
YAF snoops packets from pcap dump files or live capture, and produces bidirectional flows. These flows can be sent to IPFIX collectors, or be stored in an IPFIX-derived file format.
VERMONT (VERsatile MONitoring Toolkit)
A reference implementation of the IPFIX and PSAMP protocols developed as part of the HISTORY project at the German universities of Erlangen and Tübingen, and of the European DIADEM Firewall project.
libipfix
A C library that implements the IPFIX protocol.
libfixbuf
Aims to be a compliant implementation of the IPFIX protocol message format, from which fully compliant IPFIX Collecting Processes and IPFIX Exporting Processes may be built. In addition of the IPFIX Protocol, libfixbuf supports efficient persistent storage of IPFIX data using the method outlined in draft-trammell-ipfix-file-NN.
NetSA Aggregated Flow (NAF) toolchain
Tools for creating and analyzing timeslice-organized bidirectional flow files in the IPFIX-inspired NAF format.
FlowScan
A Perl-based system to analyze and report on flows collected by flow-tools, lfapd or cflowd, by Dave Plonka. Sample output graphs are available too, as well as Majordomo-driven mailing lists for announcements and general discussion (archive). It is currently built on Cflow.pm. User-contributed tools based on FlowScan include:
CarrierIn from Stanislav Sinyagin
which claims to be more suitable for larger ISP/Carriers
CUFlow from Matt Selsky and Johan M. Andersen at Columbia University
which is an alternative graphing tool "designed to combine the features of CampusIO and SubNetIO". Robert S. Galloway has contributed a nice howto-style document describing how it can be used.
FlowMonitor from Johan M. Andersen at Columbia University
monitors individual users' network usage against a bandwidth usage policy.
JKFlow by Jurgen Kobierczynski
A new reporting module which is highly configurable using an XML configuration file.
FlowScan+
An extension to FlowScan developed by KISTI/KAIST. Adds servlet-based visualization and support for queries for top user, AS, port, protocol, etc. This is supposed to be available under http://flowscan.kreonet2.net/, but that site doesn't seem to be responsive.
flow-tools
Similar to cflowd but implemented as a set of smaller tools, with the addition of compression of the recorded data, thus capable of recording many more flows in a given amount of disk space. See paper about its application for Intrusion Detection. There is also a mailing list for the package.
There is a short presentation called Ohio Gigapop Traffic Measurements that shows some examples on how flow-tools can be used.
The package is widely used, and there are quite a few user contributions, such as
FlowViewer
Web-interface to flow-tools. Consists of three tools: FlowViewer provides the user with web access to many of the textual and statistical flow-tools reports. FlowGrapher provides a web page with a graph of the selected flow data. These web pages can be saved. FlowTracker (introduced in FlowViewer 3.0, released in July 2006) allows the user to maintain this information long-term by creating four MRTG-like graphs. Filtered flow data is collected every five minutes and the graphs are updated. FlowTracker requires Tobi Oetiker's RRDtool package. Screenshots are available.
flow-extract
which can be used to filter flow-tools-recorded flows through user-specified tests
a set of "Inter.netPH contribs"
by Horatio B. Bogbindero
some patches and a Python module
by Robin Sommer.
flow-pairs
A script that extracts lists of the highest bandwidth consumers by host and by port. Installed at UCB. Seems to have similar uses as the older MATHE system.
Net::Flow NEW:
Perl module for de- and encoding Netflow (v5/v9) and IPFIX packets.
jflow
A set of Java classes for collecting and analyzing NetFlow data. Supports Netflow versions 5 and 6, multithreaded implementation to facilitate real-time traffic accounting and analysis.
Autofocus
A traffic analysis and visualization tool that describes the traffic mix of a link through textual reports and time series plots. The underlying research is documented in a SIGCOMM 2003 paper, Automatically Inferring Patterns of Resource Consumption in Network Traffic, C. Estan, S. Savage, G. Varghese (PDF paper, PPT slides).
Wisconsin Netpy
Netpy is a network traffic analysis and visualization package developed at University of Wisconsin-Madison. This application is intended for the use of network administrators and it can help understand usage trends in your network as well as support interactive analysis of specific network events of interest. Netpy is distributed under GPL and a BDS-like license. Netpy stores NetFlow records in a local database after applying some sampling to reduce the size of the data. The analysis engine supports interactive analyses on this data where the user chooses the time interval of interest, the filtering rules to apply to the traffic and the type of analysis. The netpy console allows the user to manage the database, and perform analyses interactively or through scripts. The graphical user interface visualizes the results of the analyses accessing the database locally or remotely through a netpy server that is also part of the package.
Stager
Stager is a system for aggregation and presentation of network statistics from the flow-tools package. Includes PostgreSQL storage of aggregated statistics, as well as a Web frontend. A public demo is available.
nfstat
Developed to analyze (sampled) Netflow data from the Internet2 Abilene backbone. This is used to generate the Internet2 NetFlow Weekly Reports, which contain interesting statistics not easily found elsewhere, such as distribution of bulk flow throughput. There are two mailing lists for announcements and for user discussions, respectively.
CAIDA cflowd
Rather complex system with distributed log servers. Released in 1998, this was the first open-source software system to work on NetFlow data, but doesn't seem to be maintained anymore. CAIDA have prepared a nice FAQ which contains interesting information both on Cflowd and on NetFlow in general. CAIDA has announced that they no longer support cflowd, and recommend that people move to flow-tools instead.
Aflow
Small Netflow monitoring tool developed by ARIN, available under GPL. Features include easy configuration, maintenance of and graph generation from RRDtool files, pf/tcpdump-style filter rules. There is a mailing list for announcements and discussion.
ASFLOW (already missing in action?)
Tool to analyze traffic to "would-be" BGP neighbors. Presented by Richard Steenbergen and Nathan Patrick at NANOG 35, October 2005. There is supposed to be both an easy-to-use Perl version and a high-performance (but somewhat complex) C version.
Fluxoscope
Software used for charging, monitoring, and traffic analysis at SWITCH. Includes its own NetFlow v5 accounting receiver which aggregates traffic into multidimensional matrices (AS/site/application). Most of the software is written in Common Lisp.
UDP Samplicator
A small program that receives UDP datagrams and redistributes them to a set of receivers. Useful to distribute NetFlow accounting streams to multiple post-processing programs. Is able to distribute only a specified percentage of all packets to each receiver. Note that recent versions added the possibility of ``spoofing'' the original sender's IP address.
Anonymization Application Programming Interface (AAPI)/AnonTool
An open-source implementation of Anonymization API. Includes a set of ready-to-use applications for anonymization of Netflow (v5 and v9), as well as PCAP traces.
CANINE
"A NetFlows Conversion/Anonymization Tool for Format Interoperability and Secure Sharing". Converts NetFlow data between various formats including NetFlow v5 and v7, NFDUMP, CiscoNCSA and ArgusNCSA, and is able to apply various methods of anonymization based on user configuration. See also the FlowCon 2005 paper by K. Luo, Y. Li, A. Slagell, and W. Yurick.
Panoptis
An open-source project started in 2001 by Costas Kotsokalis of GRNET. Uses NetFlow accounting data to detect (Distributed) Denial of Service attacks. Status as of November 2006: Supports NetFlow v1, v5 and v8 (router-aggregated) (with v8 untested for its biggest part). The system supports proof-of-concept attack trace-back using a mesh of detectors. Updates have been introduced so that the project compiles on newer systems.
Flamingo
Real-time 3D traffic visualization system developed at Merit. This client/server system based on Netflow and OpenGL plots traffic patterns by IP address, AS, or port numbers, and allows interactive exploration of this data. Sample graphics and a paper are available from the Website.
MHTG (Multi Host Traffic Grapher)
Uses NetFlow to generate per-host graphs of traffic for a campus network. Nice user interface implemented as a Java applet which allows interaction with traffic plots. The software consists of a C++ program to process NetFlow data, a Mysql backend, and Perl frontend and the Java grapher.
Matt's Quick & Dirty CFLOWD tutorial and scripts...
Postprocessing scripts for cflowd data by Matthew Petach
flow2rrd.pl
Converts a cisco NetFlow stream into set of RRDtool files, based on set of IP netmasks. By Alex Pilosov.
bmpcount
A library of bitmap counting algorithms that count the number of active flows in a network traffic trace. To be able to use it, you should be familiar with the paper that describes the algorithms it implements: _Bitmap algorithms for counting active flows on high speed links_, C. Estan, G. Varghese, M. Fisk, Internet Measurement Conference 2003 (PDF paper, PPT slides)
Slate
An application that converts LFAP data into NetFlow records - see http://www.nmops.org/.
Ntop
This well-known libpcap-based network usage monitor has been extended to produce NetFlow v5 accounting data. It also supports sFlow.
SiLK
SiLK, the System for Internet-Level Knowledge, is a collection of netflow tools developed by the CERT/NetSA (Network Situational Awareness) Team to facilitate security analysis in large networks. The toolset includes programs such as rwfilter, rwcount, rwuniq. There are plans to develop this further into an "Analyst's Desktop", described in a FloCon'05 paper, R: A Proposed Analysis and Visualization Environment for Network Security Data, J. McNutt (PDF). (Ed.: Should this be "RAVE: A Proposed..."?) The idea is to base this on the R statistical programming language (see www.r-project.org), which supports exploratory data analysis well.
Java Netflow Collect-Analyzer
Collects Netflow v1/v5/v7/v8/v9 packets from Cisco/Juniper routers or nProbe. It can store both raw data or analyzed contents to a database using JDBC.
UPFrame
This UDP/Netflow Processing Framework is a system for real-time processing of UDP packet streams such as Netflow export data. It features a general infrastructure for dynamically configurable plugin modules.
nProbe
A small self-contained program that generates NetFlow accounting data for a traffic stream sniffed off one or several interfaces. Works under Unix and Windows environments. It can be used to build inexpensive NetFlow probes.
fprobe (I)
Traffic probe that can generate NetFlow data. Based on the libpcap library. Fairly small implementation in C.
fprobe (II)
Another NetFlow-generating software traffic probe.
Softflowd
Traffic probe that can generate NetFlow data. Based on libpcap. Comes with a NetFlow collector in Perl. Both the server (probe) and client (collector) support export/import over IPv6. Very lean (as of June 2004) implementation in C.
The pfflowd variant is based on OpenBSD's PF interface.
The flowd companion NetFlow collector includes features such as multicast, IPv6 and NetFlow v9 support, as well as fast upfront filtering.
Argus from QoSient
This network Audit Record Generation and Utilization System can be used for intrusion detection and QoS monitoring. It is also mentioned in the reference section of these pages.
RENETCOL (RENATER Network Collector)
GPL'ed Netflow collector with support for Netflow v9, IPv6, Multicast, and MPLS.
Flowc
"a tool for gathering, storing and analyzing traffic accounting for Cisco routers with NetFlow enabled switching (version 5). This package could be used by ISP for planning, analysis and billing procedures."
CESNET NetFlow Monitor
by Jan Nejman.
RUS-CERT tools
The CERT of the Stuttgart University computing center (RUS-CERT) has published some tools that they use internally to analyze Netflow data. Some of the documentation is in German.
pmacct
A set of tools to account and aggregate IP traffic. Supports libpcap, Netflow v1/v5/v7/v8/v9, and sFlow v2/v4/v5 for both IPv4 and IPv6 traffic.
NEye
NEye is a Netflow V5 collector. It logs incoming Netflow V5 data to ASCII, MySQL, or SQLite databases, and it makes full use of POSIX threads if available. It works on most major platforms (Linux, Solaris, AIX, Irix, HP/UX, Mac OS X, Digital Unix, etc.) and older ones too (Ultrix, Nextstep, etc.).
NetFlow2MySQL, NetFlow2XML, and pcNetFlow
Three products from a research project at the NARA Institute of Science and Technology.
F.L.A.V.I.O. (see also the FreshMeat page)
A Perl-based NetFlow collector that stores flow data "into a MySQL database and gets it back to graph daily, weekly, monthly and yearly charts."
NetFlowMet
Starting with release 4.2, Nevil Brownlee's NeTraMet package includes NetFlowMet, which implements an RTFM meter fed on Netflow accounting data.
NetFlow Accounting software from ABPSoft
A self-contained NetFlow processing system written in C. Writes captured flows to file. Postprocessor breaks up this data over peers according to a definition file.
EHNT (Extreme Happy NetFlow Tool) by Nik Weidenbacher
Another self-contained NetFlow accounting packet processor. The receiving process also functions as a server to which various kinds of clients can connect. Also written in C.
Hendrik Visage's NetFlow tools
FTP site with various tools for NetFlow postprocessing. In particular, you will find:
  1. a UDP duplicator (hack of samplicator to preserve the source router IP)
  2. a couple of hacks to cflowd for dumping the flows every %n seconds as well as a "flhh" to output flowdump stuff aggregated, ready for a `grep|sed "s/../update  /"|rrdtool -`
netMET - Network's METrology
Network measurement solution for the French regional academic networking community, developed at the C.I.R.I.L in Nancy. Includes an HTML interface and support for accounting and security monitoring.
MATHE
An article (in French) about a Netflow accounting and visualization system used at EPFL. Uses an Oracle database and Perl DBI/GD scripts to generate a nice breakdown of external traffic to departments/institutes.
JANET Traffic Accounting Site
An impressive application of Netflow which is used for volume-based charging for JANET's U.S. connection. Other statistics at JANET were done using NeTraMet.
InMon sFlow Toolkit
Open source tools for analyzing sFlow data. Allows sFlow data to be used with a number of open source tools, including: tcpdump, snort and MRTG or rrdtool. Also able to convert sFlow packets to NetFlow packets.
Net::sFlow
Perl module to parse sFlow messages. Written by Elisa Jasinska from AMS-IX as a basis of the sFlow-based traffic analysis service for AMS-IX members. The use of this at AMS-IX has been described in presentations and a paper, links to which can be found in the references section.
Webview Netflow Reporter
Webview Netflow Reporter is an enterprise-focused Netflow reporter/analyzer tool featuring clickable graphs, powerful categorization that goes beyond simple TCP/UDP port names, automatic exporter discovery, and full access to all aspects of the raw flow data (millisecond accuracy, QoS settings, TCP flags, etc). It uses flow-tools and/or flowd as a collector.

Commercial Applications

Andrisoft WANGuard
The Andrisoft WANGuard Platform relies on NetFlow v.5 or Port Mirroring / SPAN to provide in-depth network traffic analysis and DDoS detection and mitigation. It can be used to generate traffic graphs and traffic accounting reports per IP, per subnet, per IP Zone or per router interface / switch port.
Watch4net APG (Automated Performance Grapher)
APG is a reporting tool that provides performance and capacity reports on network, servers, applications and Netflow data.
Apogee Networks
The NetCountant network usage-based billing system and the NetScope real-time network monitoring and performance analysis solution support NetFlow, RMON2, RADIUS, other SNMP MIBs, and ``Layer 7'' application/content switches.
Arbor Networks
Peakflow DOS detects denial-of-service attacks, and Peakflow Traffic analyzes traffic and routing history. Both can process NetFlow accounting data. As of November 2003, Arbor is said to support Netflow v9.
Network Signature BENTO
BENTO stands for ``BGP Enabled Network Traffic Organizer'' and is a high-performance NetFlow data processor with an integrated BGP-4 implementation to facilitate traffic analysis based on complex external routing relationships. Product offerings include a software/support package and an ``appliance'' consisting of a preconfigured rack-mount server.
Caligare Flow Inspector and NetImonitor
Analyzes NetFlow data for network monitoring as well as attack detection and response. Works with NetFlow data export version 1,5,6,7 and 9. NetImonitor is primarily designed for use in the United States.
Cisco
NetFlow FlowCollector/Network Data Analyzer
Similar to cflowd but productized, with a (Java-based) GUI and possibly better possibilities of defining filters and aggregation schemes.
Cisco NAM (Network Analyzer Module)
This is a "NetFlow collector on a linecard" for the Catalyst 6500/7600 OSR platform.
Concord
Network Health uses NetFlow and RMON2 accounting information ``to determine application, bandwitdth and server usage.''
FlowFe NEW:
FlowFe is a Netflow v5 and v9 collector and front-end with an SQL backend for accurate real-time and historical reporting. It also has the ability to save reports as PDFs for archival purposes.
Crannog Software's (now Fluke Networks) Netflow Monitor
LAN and WAN bandwidth analysis based on NetFlow data. Includes a Web interface including Java applets to display traffic graphs and to enable drill-down. Runs on Microsoft Windows NT4/2000/XP and on Unix. Evaluation version of NetFlow Live available. Note that Crannog has been acquired by Fluke Networks in January 2007, and rebranded this product as NetFlow Tracker.
Cyclades-nQuirer
A network traffic monitoring appliance that can generate data in both Netflow and nTop formats.
Digiquant
IMS accounting and billing system based on Oracle 9i under Unix.
Gadgets Software & Professional Services Ltd.
Network Intelligence traffic measurement and visualisation software for GNU/Linux and Windows (client only) platforms. Free trial available. Includes 3D visualization using OpenGL.
The author also wrote bbnfc, a ``bare-bones Netflow collector tool'' that simply receives and displayes Netflow v5 packets.
Hewlett-Packard
The Smart Internet Billing Solution usage management system and well as OpenView Performance Insight for Networks (OVPI) use NetFlow accounting data as possible input.
Infosim StableNet - Performance Management Engine
StableNet PME provides End-to-End (E2E) Service Level Management (SLM) by monitoring and reporting on the systems, networks and applications. StableNet supports the following flow technologies out of the box: Netflow, cFlow, sFlow, Netstream.
InfoVista Corporation
InfoVista Service Level Management (SLM) and conformance solution.
InMon Traffic Sentinel
is a commercial, web-based application running on Linux that provides real-time and historical analysis of flow information from NetFlow, sFlow, LFAP or HP Extended RMON sources. Web queries provide easy access to historical traffic matrices. Real-time top talker charts identify sources of congestion. Includes network-wide threshold and alert features as well as anomaly detection.
IsarFlow from IsarNet
IsarFlow is a traffic analysis tool for accounting, capacity planning, QoS monitoring, and application distribution within Citrix sessions based on Netflow.
Ixia
IxTraffic integrates NetFlow accounting data with topology information from a live BGP-4 feed to allow analysis of inter-domain traffic patterns.
Lancope StealthWatch
Flow-based Network Behavior Analysis appliance with advanced user identity tracking. Can handle Netflow and sFlow data, or capture packets from mirrored ports.
LoriotPro
A network monitoring ("supervision" in franglais) system that includes a Netflow plugin. Stores flow data in a MySQL database.
ManageEngine NetFlow Analyzer
Netflow-based bandwidth monitoring tool from AdventNet. Supports location of bottlenecks and allows drilling down to find traffic that is causing them. Thirty-day evaluation license available free of charge. Versions for Windows and Linux (x86).
Mazu Networks
Mazu Profiler analyzes and models enterprise network traffic. It provides visibility into network behavior, protects against worms and other malware, and supports auditing and policy enforcement. It supports Netflow v1/5/7/9 as well as other data collection mechanisms.
Micromuse
Cisco Info Center USM ``acquires, analyzes, displays and exports Internet usage data.'' Note that Micromuse was integrated into IBM under the "IBM Tivoli Netcool" brand.
NARUS
OSS Mediation solutions. They also do anomaly detection.
Nazca.Billing
Integrated billing software for "Telephony, Internet and Networks". Contains interfaces to many accounting systems including NetFlow.
NetQoS ReporterAnalyzer
Scalable solution for network capacity planning, troubleshooting, and traffic analysis, including traffic visualization capabilities.
NetReflex by Guavus NEW:
Network-wide analytics and anomaly detection platform. The system fuses traffic and routing data, builds traffix matrices, and performs anomaly detection and classification.
NetUp Products
UTM is a billing system for ISPs. It can use Netflow (v5) and several other accounting methods. It supports a rich variety of charging and payment schemes.
NDSAD Traffic Collector is an open-source (GPL'ed) tool that captures packets and generates a Netflow (v5) accounting stream.
NetUsage from Apoapsis (formerly called WANBUS)
The NetUsage suite strives to provide visibility of network traffic, producing meaningful reports not only for network professionals, but for IT management, business managers and accounts departments. Supports network traffic monitoring, capacity planning, business justification and cost control.
SolarWinds Orion NetFlow Traffic Analyzer
Windows-based commercial system that stores NetFlow data, generates various types of charts, and provides "drill-down" capabilities.
PRTG Traffic Manager
Windows-based bandwidth management software from Paessler. Uses SNMP, Netflow, and packet capture for monitoring and classifying bandwidth usage. Besides the commercial license, there is also a (restricted) "freeware" license.
QRadar from Q1 Labs
The system can use Netflow data, but also includes its own payload-aware flow collector which produces bi-directional flow information in a format called QFlow. Includes anomaly detection.
Plixer Scrutinizer NetFlow Analyzer
NetFlow-based Enterprise-level traffic analysis tool with GUI-based reporting (topN hosts/applications etc.) and zoom/drill-down. Uses MySQL back-end. Free (as in free beer) edition available.
I-ABA and M-NTM from Tek Yazilim
Windows-based software to analyze NetFlow (and Cisco IP Accounting) statistics. I-ABA specifically analyzes AS-to-AS traffic streams. Trial versions can be downloaded.
Quallaby
Has a Netflow Application Pack for its PROVISO system for network performance monitoring and service assurance. Quallaby was acquired by Micromuse, which was itself acquired by IBM. The Netflow Application Pack is maintained in the 4.4.1 release and supports Netflow versions up to v8.
NetScout
nGenius Performance Manager ``is a complete solution for proactive monitoring, troubleshooting, capacity planning, and Voice over IP (VoIP) monitoring''.
Portal Software
Infranet real-time customer management and billing software.
RODOPI
Billing software for ISPs.
XACCT
Commercial vendor of accounting and billing solutions with the ability to process (among others) Netflow accounting data