So, after being ill for almost a week, and that continuing down the drain, I slowly start to feel better. My stomach was sore for more then a week, and I am periodically getting stalked by headaches which aren’t a real pleasure. But slowly they appear to be slowing down and putting me back into shape again. I started translating again and wrote some scripts that I needed to write for JR-Hosting, so progress is upcoming again. In the future night and tommorrow where possible I will continue scripting, translating and finding a few low hanging fruits for FreeBSD so that I can show that I am still alive. I am a regular slacker lately and that’s not something I actually like. So… lets get this ball rolling again!
Today I made the step to move from Apache to nginx. While Apache had been serving me for ages, I think it’s time for something different. nginx does precisely what I need at this point, without dreaded much overweight that Apache has. All but two vhosts had been moved over to the nginx installation at this moment. One of them will be converted today or tommorrow, the other one might not be done at all, but proxied through nginx. The problem with that host is that it’s doing Subversion hosting, and I have yet to find a proper http(s) version that supports nginx
That said; I will ofcourse keep running Apache in various places and keep track of things, but my own masterpiece is now nginx.
Release 2.0.a2 (Alpha quality) for check_honeynet is here
, go check it out!
-
#!/usr/bin/perl
-
# $Id: check_honeynet.pl 801 2008-06-03 20:40:53Z remko $
-
###########################################################################
-
-
###########################################################################
-
# Copyright (C) 2005-2008, Remko Lodder <remko@FreeBSD.org>. All rights reserved.
-
#
-
# Redistribution and use in source and binary forms, with or without
-
# modification, are permitted provided that the following conditions
-
# are met:
-
# 1. Redistributions of source code must retain the above copyright
-
# notice, this list of conditions and the following disclaimer.
-
# 2. Redistributions in binary form must reproduce the above copyright
-
# notice, this list of conditions and the following disclaimer in the
-
# documentation and/or other materials provided with the distribution.
-
#
-
# THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND
-
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-
# ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
-
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
-
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
-
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
-
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
-
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
-
# SUCH DAMAGE.
-
###########################################################################
-
-
###########################################################################
-
# Contributors:
-
# Ivo Naninck, (Language), Marc Plaisier (language), Mart vd Wege (Mailer
-
# suggestion),
-
# Lance Spitzner (the most valueable feedback regarding the script).
-
# Richard Arends (suggested the actual perl mailer; and gave a good example
-
# on how to use the module),
-
# Robert Blacquiere (showed me Simple::Config, which I am using now to
-
# obtain configuration informatin, as well as write statistics to a
-
# temporary file)
-
#
-
# Script:
-
# This is a stand alone script written for honeynet.org. It's purpose is
-
# to check the listed mirrors to see which one is outdated and notify
-
# the administrator of the outdated mirror if needed. This way the
-
# mirrors will always be within a certain timeframe and can be removed
-
# if they are too outdated.
-
#
-
# Written in colaboration with Lance Spitzner <lance@honeynet.org>
-
###########################################################################
-
-
###########################################################################
-
# Differences between releases, started this between 1.5 and 1.6 sorry
-
# for the loss of potential usefull information (although I might be able
-
# to retrieve the missing data from my CVS branches).
-
# Note that all the three version based versions are just bugfixes to some
-
# degree.
-
# Version Who What
-
# 1.1 Remko Initial script to check the mirrors.
-
# 1.1->1.2 Remko
-
# 1.2->1.3 Remko
-
# 1.3->1.4 Remko
-
# 1.4->1.5 Remko Code cleanups, restructure of code; corrected
-
# some bugs between various releases spotted
-
# by Lance.
-
# 1.5->1.6 Remko Cleanups, changed structure for timestamp
-
# retrieval; make it human readable and match
-
# on a specific pattern. Made the version dependend
-
# on the configuration version and visa-versa
-
# to be able to make big config changes.
-
# BF-1: Fix the email send options by using the
-
# correct Net::SMTP commands.
-
# BF-1: Fix the parsing of the new mirrorprobe
-
# layout.
-
# BF-2: Change the Mailer used to send out the
-
# report.
-
# 1.6->1.7 Remko Implement Config::Simple, makes configuration
-
# much easier!
-
# 1.7->2.0 Remko a1: Implement that we can trace various sites with
-
# multiple problems more easily by using a
-
# semi-persisent configuration file. This brings
-
# the entire branch to an entirely new world.
-
# Which is why we bump the version to 2.0
-
# a2: Implement file-statistics checker, if the
-
# file does not exist yet, we need to make sure
-
# we know and that we can bypass certain readins
-
# so that we are not going to get into trouble
-
# later on.
-
###########################################################################
-
-
use strict;
-
use warnings;
-
use LWP::Simple;
-
use Getopt::Std;
-
use Mail::Sendmail;
-
use Config::Simple;
-
-
###########################################################################
-
# variables. All configurable options are defined below. Please adjust them
-
# to your need.
-
###########################################################################
-
-
# Version, author and script specific behaviour
-
my $author = 'Remko Lodder <remko@FreeBSD.org>'; # Name of the author
-
my $name = __FILE__; # Our scriptname.
-
my $MAJOR = '2'; # Our major version
-
my $MINOR = '0'; # Our minor version
-
my $PATCHLEVEL = 'a2'; # Our patchlevel
-
my $version = "$MAJOR.$MINOR.$PATCHLEVEL"; # Our version.
-
-
###########################################################################
-
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
-
# Do not edit anything below this line unless you know what you are doing.
-
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
-
###########################################################################
-
-
###########################################################################
-
# Prototypes, specify upfront what we are going to use if globally needed
-
# and prototype arrays, hashes and functions.
-
###########################################################################
-
-
# create the option hash, we will use that later to add our option flags.
-
my %option = ();
-
-
# Template our arrays
-
my (@MIRRORS);
-
-
# Template some standard variables
-
my ($CONFIGFILE, $status, $reporthost, $timestamp,$target,
-
$enable_mail, $cfg, $cfg_out, $outfile, $alert_max,
-
$alert_min, $alert_mid, $mail_from, $mail_bcc, $mail_cc);
-
-
# Prototype functions
-
sub fetch_data ($$);
-
sub email_report ($$$);
-
sub convert_input ($);
-
sub process_mirrors (@);
-
sub create_timestamp ($);
-
-
###########################################################################
-
# Script itself
-
###########################################################################
-
-
getopts("cC:f:F:hpR:tv", \%option);
-
-
my $config_flag = 1 if $option{f};
-
my $config_option = $option{f};
-
-
my $create_flag = 1 if $option{c};
-
my $help_flag = 1 if $option{h};
-
my $process_flag = 1 if $option{p};
-
my $test_flag = 1 if $option{t};
-
my $convert_flag = 1 if $option{C};
-
my $fetch_flag = 1 if $option{F};
-
my $retrieve_flag = 1 if $option{R};
-
-
# Simply assign the contents of the configuration parameter to the
-
# configuration file variable, otherwise overrule it with the
-
# default value
-
-
$CONFIGFILE = $config_option || "./honeynet.cf";
-
-
# Read in current configuration.
-
# This should always succeed, in case it doesn't then we cannot do
-
# anything at all.
-
-
$cfg = new Config::Simple($CONFIGFILE);
-
-
#####
-
##### Global variable assignment after reading config
-
#####
-
-
# Read in some variales upfront, so that we can more
-
# easily use them later on.
-
-
@MIRRORS = $cfg->param('CONFIG.mirrors');
-
$outfile = $cfg->param('CONFIG.outfile');
-
$enable_mail = $cfg->param('CONFIG.enable_mail');
-
$alert_max = $cfg->param('CONFIG.alert_max');
-
$alert_mid = $cfg->param('CONFIG.alert_mid');
-
$alert_min = $cfg->param('CONFIG.alert_min');
-
$mail_from = $cfg->param('CONFIG.fromaddr');
-
$mail_bcc = $cfg->param('CONFIG.ccaddr');
-
$mail_cc = $cfg->param('CONFIG.mirroradmin');
-
-
if (-e "$outfile")
-
{
-
-
# Read in the statistics from our last run
-
$cfg_out = new Config::Simple($outfile);
-
-
# Make an instant backup before we do a new run.
-
$cfg_out->write("$outfile.orig");
-
}
-
else
-
{
-
# Create the statistics file for our entire run
-
$cfg_out = new Config::Simple(syntax=>'ini');
-
}
-
-
# Make sure that there us a version statement in the configuration file, so that we can see whether we are
-
# compabible or not.
-
-
if (!$cfg->param('CONFIG.version'))
-
{
-
print "It appears that you do not have a version statement in your configuration file. This means that the version you are using now is
-
too old, make sure that you obtain the latest one and update that to your needs.";
-
}
-
-
if ($cfg->param('CONFIG.version'))
-
{
-
my($version_def_major,$version_def_minor) = $cfg->param('CONFIG.version') =~ /(\d+)\.(\d+)/;
-
-
# the script can defer between MAJOR and MINOR releases; bugfixes aka patchlevels are
-
# not affected by this and can thus be ignored.
-
# TODO: This check should be made more flexible in the future, version 1.4 and 1.5 share
-
# the same configuration file and should both pass.
-
-
if(($version_def_major lt $MAJOR) or ($version_def_minor lt $MINOR))
-
{
-
print "You appear to be using an older configuration file that might not be compatible with the current version of the
-
script, please validate that you have the latest options included and copy over the version statement from the latest available configuration
-
file. Make sure that the old version statement is overwritten!
-
The current script runs on version: $version, while the configuration is for version $cfg->param('CONFIG.version')\n";
-
}
-
}
-
-
# Create a new timestamp that will be fed into the mirrors, which we can use to test the
-
# age of the mirror.
-
-
if ($create_flag) {
-
create_timestamp($cfg->param('CONFIG.probefile'));
-
}
-
-
elsif ($process_flag) {
-
# Process the mirrors using the hash we have for them.
-
process_mirrors(@MIRRORS);
-
}
-
-
# test mode, printout information on screen.
-
elsif($test_flag)
-
{
-
# In test mode we dont send out emails.
-
$enable_mail = 0;
-
-
-
# Process the mirrors using the hash we have for them.
-
process_mirrors(@MIRRORS);
-
-
}
-
-
# convert input from unixtime to human readable time.
-
elsif($convert_flag)
-
{
-
}
-
-
# fetch the mirrorprobe file from the given host
-
elsif($fetch_flag)
-
{
-
my $result = fetch_data($option{F}, $cfg->param('CONFIG.sourcefile'));
-
print OUT $result;
-
}
-
-
# Fetch the mirror timestamp and parse it. Print the output back on the screen.
-
elsif($retrieve_flag)
-
{
-
my $result = fetch_data($option{R}, $cfg->param('CONFIG.sourcefile'));
-
}
-
-
# People expect a help option, provide it for them.
-
elsif ($help_flag)
-
{
-
print_help();
-
}
-
-
# No valid options had been given, fallback to the help information.
-
else
-
{
-
print_help();
-
}
-
-
# print_help: expects no input, just prints the help information on how
-
# the application should work.
-
sub print_help
-
{
-
print("Usage:\t$name [ -c ] [ -C <value> ] [ -f <configurationfile> ] [ -F <host> ] [ -h ] [ -p ] [ -R <host> ] [ -t ]
-
\t-c\tCreate the timestamp for the localmachine. This timestamp can be used to determine when the mirror was last updated.
-
\t-C\t<value> converts the unix timestamp to human readable format
-
\t-f\t<filename> Use the specified configuration file
-
\t-F\t<host> fetch the timestamp for an external host, for example: www.honeynet.nl
-
\t-h\tprint this help.
-
\t-p\tCheck the status of the mirrors, and report the output to us
-
\t-t\tTest mode, does not send out emails, but prints the information on the screen.
-
Version: $version
-
Originally written by Remko Lodder <remko\@FreeBSD.org, for the honeynet project.\n");
-
}
-
-
# create_timestamp: expects a variable filled with where we should store
-
# the unixtime (for remote mirrors).
-
sub create_timestamp ($)
-
{
-
close F_OUT;
-
}
-
-
# convert_input: expects unixtime and converts it to human readable time.
-
sub convert_input ($)
-
{
-
return $output;
-
}
-
-
# fetch_data: gets two variables as input, one with the fqdn of the remote host and one
-
# with the location of where we expect the remote file. XXX: This looks a bit ugly at the
-
# moment.
-
sub fetch_data ($$)
-
{
-
-
my $return_data = ();
-
my $data = get("http://$source$sourcefile");
-
-
# IF the remote data is present, take out the numberic time value and return that
-
# ELSE obscure the data, which will revert to 1969/1970 (depending on what the machine
-
# considers EPOCH).
-
if ($data)
-
{
-
chomp $data;
-
-
$return_data = $data;
-
$return_data =~ s/\n/\ /;
-
if ($return_data =~ /\S+ \S+ (\d+) \S+/)
-
{
-
$return_data = $1;
-
}
-
return $return_data;
-
}
-
-
else
-
{
-
$data = 0;
-
return $data;
-
}
-
}
-
-
# proccess_mirrors: gets an array as input with all the mirrors in it. It will walk through all hosts
-
# and do specific actions with them, like printing out a report, or emailing it to the remote
-
# administrator. It keeps a record of the current statustime.
-
sub process_mirrors (@) {
-
my %mirror_time;
-
my @mirror_list = @_;
-
-
for my $target(@mirror_list)
-
{
-
$mirror_time{$target} = fetch_data($target,$cfg->param('CONFIG.sourcefile'));
-
-
# We should probably dont need to use this since it is implied in the routine itself.
-
if(!$mirror_time{$target})
-
{
-
$mirror_time{$target} = 0;
-
}
-
-
# declare our local time before continueing.
-
my $mirror_ctime = $mirror_time{$target};
-
my $mirror_difftime = $honeynet_ctime - $mirror_time{$target};
-
-
# If the resulting number is less then zero, the remote host is outdated.
-
my $mirror_timediff = $cfg->param('CONFIG.timeout') - $mirror_difftime;
-
-
# Everything that has a calculated time which is bigger then the timeout is
-
# on the right track, everything that isn't, gets either notified or the
-
# output will be present on screen.
-
if ($mirror_timediff gt "0")
-
{
-
if($cfg->param('CONFIG.verbose') ne "0")
-
{
-
$status = "OK";
-
$reporthost = $target;
-
write; # Write out our template.
-
}
-
next;
-
}
-
else
-
{
-
my $info = $cfg_out->param("mirrors.$target");
-
{
-
if ($info eq $alert_max || $info eq $alert_mid || $info eq $alert_min)
-
{
-
# The email option is enabled and we found an outdated mirror
-
# jump to the email_report function.
-
if($enable_mail)
-
{
-
email_report($target,$mirror_time{$target},$info);
-
}
-
}
-
-
# Update our local statistics first before anything else
-
$cfg_out->param("mirrors.$target", $info+1);
-
}
-
else
-
{
-
# The email option is enabled and we found an outdated mirror
-
# jump to the email_report function.
-
if ($enable_mail)
-
{
-
email_report($target,$mirror_time{$target}, $alert_min);
-
}
-
-
# Begin our statistics with 1.
-
$cfg_out->param("mirrors.$target", 1);
-
}
-
-
-
if($cfg->param('CONFIG.verbose') ne "0")
-
{
-
$status = "FAIL";
-
-
# Only override if the message has the old timestamp and thus is broken.
-
if($timestamp =~ /19[6-7]\d/)
-
{
-
$timestamp = "Mirrorprobe file problems!";
-
}
-
$reporthost = $target;
-
write; # Write out our template.
-
}
-
}
-
}
-
-
# Specify how our output is going to look like. This generates a nice overview
-
# on the current status of the various mirrors. This is only being used if we
-
# print verbose.
-
# Below: HEADER
-
format STDOUT_TOP =
-
----------------------------------------------------------------------------
-
--------------------------- Honeynet Check results -------------------------
-
----------------------------------------------------------------------------
-
Status Site Last changed
-
.
-
-
# Format specifier for the content of the printout.
-
format STDOUT =
-
@<<<< @<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< @<<<<<<<<<<<<<<<<<<<<<<<<<<
-
$status, $reporthost, $timestamp
-
.
-
}
-
-
# email_report: expects a target and a mirror_time, sends out an email to
-
# the desired administrator and informs them about the current status of
-
# the mirror.
-
sub email_report ($$$)
-
{
-
my $status = ();
-
my $alert_msg = ();
-
my $recipient = $cfg->param("MIRRORS.$target") || $mail_cc;
-
-
# Make the mirror time a human parseable time, if this matches the beginning of time
-
# (For computers at least). Tell that there is a problem.
-
-
if ($mirror_time =~ /19[6-7]\d/)
-
{
-
$status = "Your mirror seems to have troubles fetching the mirrorprobe file.";
-
}
-
else
-
{
-
$status = "At this moment it seems that your mirror was last updated on $mirror_time.";
-
}
-
if ($alert eq $alert_max)
-
{
-
$alert_msg = "NOTE WELL: This is the final warning you will be receiving. Please repair the mirror as soon as possible. In the meantime we will remove your mirror.";
-
}
-
else
-
{
-
$alert_msg = "NOTE WELL: This is alert number $alert for your host. Please be advised that our current maximum is at $alert_max.\n";
-
}
-
-
my $message = "Hello $recipient,
-
You are recieving this email because your Honeynet Project mirror ($target) seems to be outdated.
-
$status
-
Could you please check whether everything is working as expected?
-
If there is something wrong or you do no longer wish to be a mirror, please notify $mail_cc
-
Thanks for supporting the Honeynet Project!
-
If you have questions or concerts, please do not hesitate to contact us!
-
$alert_msg
-
--
-
The Honeynet Project Mirror Admins";
-
-
# Fill our mailhash with information that we are going to send.
-
my %mail = (
-
To => "$recipient",
-
From => "$mail_from",
-
Bcc => "$mail_bcc",
-
# only addresses are extracted from Bcc, real names disregarded
-
Cc => "$mail_cc",
-
# Cc will appear in the header. (Bcc will not)
-
Subject => "$target seems to be outdated, please investigate",
-
'X-Mailer' => "$name $version using Mail::Sendmail version $Mail::Sendmail::VERSION",
-
message => "$message",
-
);
-
-
# Add an additional header so that we can show that this is our thing.
-
$mail{'Honeynet.org : '} = "http://www.honeynet.org";
-
-
# Send out the mail, if succesfull print that we did OK else
-
# give error.
-
if (sendmail %mail)
-
{
-
print "Mail alert sent OK.\n";
-
}
-
else
-
{
-
print "Error sending mail: $Mail::Sendmail::error \n"
-
}
-
}
-
-
# And as a final action make sure we write out our configuration file.
-
# Temporary to write out the exact config to our statistics file, later we will use this to write
-
# down our run-time statistics into a "peristent" file.
-
-
$cfg_out->write($outfile);
-
-
# End of the script.






