Output

Output can be only hwm or lwm line. If amount of events per timeframe rises above hwm, then hwm line is generated. The format of hwm line is following:

• Day Mon dd hh:mm:ss yyyy - timestamp
• hwm - fixed string
• obj - can be username or hostname or whatever user's regex matches as observation object

Similarly lwm line is generated when rate drops below lwm. Also additional statistics is included. The format of lwm line is following:

• Day Mon dd hh:mm:ss yyyy - timestamp
• lwm - fixed string
• obj - can be username or hostname or whatever user's regex matches as observation object
• max=zz - the peak amount (zz) of event in timeframe
• count=xx - count of events while rate is over hwm. May be smaller than max. Because when rate crosses hwm line, count is initalized to 1, but max at the same moment is hwm.
• duration=sec/hh:mm:ss - how long object were in hwm state. Duration is given in both forms: seconds for automation and hhmmss for humans.
• interval=hh:mm:ss - average delay between events. Equals to duration/count.

One may ask, why do i need lwm output when detecting spam? Because statistics included gives feedback about parameters to tune and also we get quick info about incident.

Know that the utility may complain about timetravel when log is not linear. In the other words, every next matched log entry must have timestamp, which is greater or equal then previous one of the same object. Unfortunately this is too common problem with logs in general.

Time Frame

... is amount of time. For example 1800 seconds (=30min). It means the utility remember events from now to 1800 seconds backwards. What happened before that is not simply interst. The interest is what is happening now? How can we achieve this? One way to do it is to remember certain amount of events and see how frequent they are. If there were more events than hwm during from now to timeframe seconds backwards we raise eyebrow and become concerned. Beaware, the shorter timeframe is the more sensitive observer becomes, the longer the timeframe is the more smoother observer becomes.

Watermarks

hwm and lwm denote High WaterMark and LowWaterMark. Those are limits which represent our tolerance towards certain activity. Hwm is used when rate increases and lwm is used when rate decreases. Why do we need two watermarks and not one? Because sometimes rate oscillates around some level and one watermark means too many output (noise). Two watermarks gives us a way to eliminate oscillation effect. Lwm must be less or equal than hwm.

Observation Objects

The configurable log matching regex gives us name of an object. It can be username, hostname, ip address or whatever thing. So one can observe many objects in one log by running instance of analyzer. If there is only one object to observe - regex must return constant.

Commandline Arguments

Following table gives us short overview of commandline arguments of throughput monitor. Every flag takes exactly one parameter. All columns are self-explaining, except maybe first. First column denotes how many times can or must be a flag specified (see under the table).

• ? - zero or one, in the other words optional argument
• 1 - exactly one, in the other words mandatory argument
• * - zero or more

Full description follows:

-d "match"

One can turn on debugging of regex matching by '-d match'. Regex tuning could be much simpler with this.

-e email

When specified, hwm and lwm messages are sent to email. If multiple emails are given with multiple -e arguments then multiple emails recieve notifications. -e option is not compatible with '-m hist'.

-hwm number

High watermark. Must be positive number. When rate rises above hwm, notify.

-i filename

Input file specification. Can be 'stdin' for stdin.

-lwm number

Low watermark. Must be positive number. When rate drops below lwm, notify. Lwm must be smaller or equal than hwm.

-m {"now"|"hist"}

'-m now' means the input file is tail'ed with system tail. '-m now' and '-i stdin' are not compatible options. '-m now' means realtime analysis, any out of frame events are immediately droppped. '-m hist' means analyzing old logs. Also note that when '-m hist' finishes, then unreported lwm events are reported with possibly future timestamps.

-ok filename

Reads in file of "ok" objects. One object per line. These objects are not checked - their activity is trusted and not object of system administrators concern. This option can be specified multiple times - all objects are used.

-tf number

Specified time frame size in seconds. Must be positive integer.

Requirements

• Perl 5.005_03
• 'mail' command or its equivalent
• 'tail' command or its equivalent
• knowledge about regex-s

Configuration

Where do i put my regex? In the beginning of the perl script there is a section 'user conf'. Adjust parameters there. Also, depending of location of perl you may need to edit first line of the script - the path of the perl.

$mail = "/usr/bin/mail";

Where your 'mail' program lives?

$tail = "/usr/bin/tail";

Where your 'tail' program lives?

$mail_subj_pfx = "$ENV{HOST}: $ENV{USER}";

Mail subject prefix. Modify this, when you want to something else than 'host: username' in the beginning of subject of mails. Alos, if you encounter 'Use of uninitialized value at ./throughput_monitor2 line 48.' at the beginning of the script output, then modify this line. It means your environment variables have a bit different names like 'HOSTNAME' instead of 'HOST' or isnt defined at all. If you dont need any prefix put leave the string empty - "".

$regex_obj = "...";

Describes object, like username, ip, etc. Its a regex. It is used for matching -ok files.

$regex_event = "...";

Describes event line in log. Be sure to include $regex_obj variable in this regex - its for your own comfort.

Regex examples

There is a predefined veriable $regex_timestamp. be sure not to modify it. Also it gives more readeble event-matchig regexes.

Postfix, spam. Following shows how to detect spam from postfix logs. Although postfix has many tools to restrict spam, it cannot detect and avoid all of it. Here are log entry to matched and regex-s to match it.