Pretium Telecom… New version check honeynet script
Jul 30

Honeynet check mirror script 1.6.2

Downloads, Technical Add comments
    Change language to :

Well well, another quick update following shortly after the previous one. It appears that the previous setup was not good enough to warrant the release actually. There were some flaws in 1.6.1 that prevented the mail option from being used. Lance reported these errors to me and I could not easily fix them. (I thought I did well but apparently not). I now decided to take the alternative route, already suggested by Mart vd Wege in 2006;
use a different Perl module to sendout the emails; and wunderbarr that appears to be working out of the box. At least on my machine.

So I did something evil; normally this should have been delivered to 1.7.0 instead of 1.6.2 but, since it is a bug and I wish to do other things for 1.7.0 or perhaps even 2.0.0 I implemented this early on in the current game (it still follows the new tradition for version numbering etc). For this to work you actually need the following script and configuration file which can be used to well checkout mirrors :-).

If you encounter any issue please let me know!

PLAIN TEXT
PERL:
  1. #!/usr/bin/perl
  2. ###########################################################################
  3. # $Id: check_honeynet.pl,v 1.45 2007/07/30 16:36:00 remko Exp $
  4. ###########################################################################
  5.  
  6. ###########################################################################
  7. # Copyright (C) 2005-2007, Remko Lodder <remko at FreeBSD dot org>. All rights reserved.
  8. #
  9. # Redistribution and use in source and binary forms, with or without
  10. # modification, are permitted provided that the following conditions
  11. # are met:
  12. # 1. Redistributions of source code must retain the above copyright
  13. #    notice, this list of conditions and the following disclaimer.
  14. # 2. Redistributions in binary form must reproduce the above copyright
  15. #    notice, this list of conditions and the following disclaimer in the
  16. #    documentation and/or other materials provided with the distribution.
  17. #
  18. # THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND
  19. # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  20. # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  21. # ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
  22. # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
  23. # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
  24. # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
  25. # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
  26. # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
  27. # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  28. # SUCH DAMAGE.
  29. ###########################################################################
  30.  
  31. ###########################################################################
  32. # Contributors:
  33. # Ivo Naninck, (Language)
  34. # Marc Plaisier, (Language)
  35. # Mart van de Wege, (Perl mailer suggestion)
  36. # Lance Spitzner (the most valueable feedback regarding the script).
  37. #
  38. # Script:
  39. # This is a stand alone script written for honeynet.org. It's purpose is
  40. # to check the listed mirrors to see which one is outdated and notify
  41. # the administrator of the outdated mirror if needed. This way the
  42. # mirrors will always be within a certain timeframe and can be removed
  43. # if they are too outdated.
  44. #
  45. # A while ago Mart vd Wege told me that a higher level mail module would be
  46. # much easier to cope with. This had been implemented as of version 1.6.2
  47. #
  48. # Written in colaboration with Lance Spitzner <lance at honeynet dot org>
  49. ###########################################################################
  50.  
  51. ###########################################################################
  52. # Differences between releases, started this between 1.5 and 1.6 sorry
  53. # for the loss of potential usefull information (although I might be able
  54. # to retrieve the missing data from my CVS branches).
  55. # Note that all the three version based versions are just bugfixes to some
  56. # degree.
  57. # Version       Who             What
  58. # 1.1           Remko           Initial script to check the mirrors.
  59. # 1.1->1.2      Remko
  60. # 1.2->1.3      Remko
  61. # 1.3->1.4      Remko
  62. # 1.4->1.5      Remko           Code cleanups, restructure of code; corrected
  63. #                               some bugs between various releases spotted
  64. #                               by Lance.
  65. # 1.5->1.6      Remko           Cleanups, changed structure for timestamp
  66. #                               retrieval; make it human readable and match
  67. #                               on a specific pattern. Made the version dependend
  68. #                               on the configuration version and visa-versa
  69. #                               to be able to make big config changes.
  70. #                               BF-1: Fix the email send options by using the
  71. #                                     correct Net::SMTP commands.
  72. #                               BF-1: Fix the parsing of the new mirrorprobe
  73. #                                     layout.
  74. #                               BF-2: Change the Mailer used to send out the
  75. #                               report.
  76. ###########################################################################
  77.  
  78. use strict;
  79. use warnings;
  80. use LWP::Simple;
  81. use Getopt::Std;
  82. use Mail::Sendmail;
  83.  
  84. ###########################################################################
  85. # variables. All configurable options are defined below. Please adjust them
  86. # to your need.
  87. ###########################################################################
  88.  
  89. # create three hashes which can be used to read our config variables from the
  90. # configuration file. Devided between version info, configuration items and mirror     
  91. # information.
  92.  
  93. my %VERSION             = ();
  94. my %CONFIG              = ();
  95. my %MIRRORS             = ();
  96.  
  97. # create the option hash, we will use that later to add our option flags.
  98. my %option              = ();
  99.  
  100. # Template some standard variables
  101. my ($CONFIGFILE, $status, $reporthost, $timestamp,$target);
  102.  
  103. # Prototype functions
  104. sub read_conf ($);
  105. sub fetch_data ($$);
  106. sub email_report ($$);
  107. sub convert_input ($);
  108. sub process_mirrors (%);
  109. sub create_timestamp ($);
  110.  
  111. # Version, author and script specific behaviour
  112. my $author              = 'Remko Lodder <remko at FreeBSD dot org>';    # Name of the author, respect the license.
  113. my $name                = __FILE__;                             # Our scriptname.
  114. my $MAJOR               = '1';                                  # Our major version number (ma.mi.pa)
  115. my $MINOR               = '6';                                  # Our minor version number
  116. my $PATCHLEVEL          = '2';                                  # Our patchlevel
  117. my $version             = "$MAJOR.$MINOR.$PATCHLEVEL";          # Our version.
  118.  
  119. ###########################################################################
  120. # Do not edit anything below this line unless you know what you are doing.
  121. ###########################################################################
  122.  
  123. getopts("cC:f:F:hpR:t", \%option);
  124.  
  125. my $config_flag   = 1 if $option{f};
  126. my $config_option = $option{f};
  127.  
  128. my $create_flag   = 1 if $option{c};
  129. my $help_flag     = 1 if $option{h};
  130. my $process_flag  = 1 if $option{p};
  131. my $test_flag     = 1 if $option{t};
  132. my $convert_flag  = 1 if $option{C};
  133. my $fetch_flag    = 1 if $option{F};
  134. my $retrieve_flag = 1 if $option{R};
  135.  
  136. # Simply assign the contents of the configuration parameter to the
  137. # configuration file variable, otherwise overrule it with the
  138. # default value
  139.  
  140. $CONFIGFILE = $config_option || "./honeynet.cf";
  141. &read_conf($CONFIGFILE);
  142.  
  143. # Make sure that there us a version statement in the configuration file, so that we can see whether we are
  144. # compabible or not.
  145.  
  146. if (!$VERSION{'version'})
  147. {
  148.         print "It appears that you do not have a version statement in your configuration file. This means that the version you are using now is
  149. too old, make sure that you obtain the latest one and update that to your needs.";
  150.         exit(1);
  151. }
  152.  
  153. if ($VERSION{'version'})
  154. {
  155.         my($version_def_major,$version_def_minor) = $VERSION{'version'} =~ /(\d+)\.(\d+)/;
  156.  
  157.         # the script can defer between MAJOR and MINOR releases; bugfixes aka patchlevels are
  158.         # not affected by this and can thus be ignored.
  159.         # TODO: This check should be made more flexible in the future, version 1.4 and 1.5 share
  160.         # the same configuration file and should both pass.
  161.  
  162.         if(($version_def_major lt $MAJOR) or ($version_def_minor lt $MINOR))
  163.         {
  164.                 print "You appear to be using an older configuration file that might not be compatible with the current version of the
  165. script, please validate that you have the latest options included and copy over the version statement from the latest available configuration
  166. file. Make sure that the old version statement is overwritten!
  167. The current script runs on version: $version, while the configuration is for version $VERSION{'version'}\n";
  168.                 exit(1);
  169.         }
  170. }
  171.  
  172. # Create a new timestamp that will be fed into the mirrors, which we can use to test the
  173. # age of the mirror.
  174.  
  175. if ($create_flag) {
  176.         create_timestamp($CONFIG{probefile});
  177. }
  178.  
  179. elsif ($process_flag) {
  180.         # Process the mirrors using the hash we have for them.
  181.         process_mirrors(%MIRRORS);
  182. }
  183.  
  184. # test mode, printout information on screen.
  185. elsif($test_flag)
  186. {
  187.         # In test mode we dont send out emails.
  188.         $CONFIG{'enable_mail'} = 0;
  189.  
  190.         print("$name: Starting\n");
  191.         print("$name: Processing mirrors\n");
  192.  
  193.         # Process the mirrors using the hash we have for them.
  194.         process_mirrors(%MIRRORS);
  195.  
  196.         print("$name: Finishing\n");
  197. }
  198.  
  199. # convert input from unixtime to human readable time.
  200. elsif($convert_flag)
  201. {
  202.         print "$option{C} resolves to " . convert_input($option{C}) . "\n";
  203. }
  204.  
  205. # fetch the mirrorprobe file from the given host
  206. elsif($fetch_flag)
  207. {
  208.         my $result = fetch_data($option{F}, $CONFIG{'sourcefile'});
  209.         open(OUT, "> $CONFIG{'outdir'}/$option{F}.timestamp");
  210.                 print OUT $result;
  211.         close(OUT);
  212. }
  213.  
  214. # Fetch the mirror timestamp and parse it. Print the output back on the screen.
  215. elsif($retrieve_flag)
  216. {
  217.         my $result = fetch_data($option{R}, $CONFIG{'sourcefile'});
  218.         print("$option{R} was last modified " . convert_input($result) . "\n");
  219. }
  220.  
  221. # People expect a help option, provide it for them.
  222. elsif ($help_flag)
  223. {
  224.         print_help();
  225. }
  226.  
  227. # No valid options had been given, fallback to the help information.
  228. else
  229. {
  230.         print_help();
  231. }
  232.  
  233. sub print_help
  234. {
  235.         print("Usage:\t$name [-c] [-C <value>] [-f <configurationfile>] [-F <host>] [-h] [-p] [-R <host>] [-t]
  236. \t-c\tCreate the timestamp for the localmachine. This timestamp can be used to determine when the mirror was last updated.
  237. \t-C\t<value> converts the unix timestamp to human readable format
  238. \t-f\t<filename> Use the specified configuration file
  239. \t-F\t<host> fetch the timestamp for an external host, for example: www.honeynet.nl
  240. \t-h\tprint this help.
  241. \t-p\tCheck the status of the mirrors, and report the output to us
  242. \t-t\tTest mode, does not send out emails, but prints the information on the screen.
  243. Version: $version
  244. Originally written by Remko Lodder <remko\@FreeBSD.org, for the honeynet project.\n");
  245. }
  246.  
  247. sub create_timestamp ($)
  248. {
  249.         my $probefile = shift;
  250.         open(F_OUT, "> $probefile");
  251.                 print F_OUT "Mirrorprobe time: " . time() . "
  252. Local time: " . convert_input(time());
  253.         close F_OUT;
  254. }
  255.  
  256. sub convert_input ($)
  257. {
  258.         my $output      = scalar localtime(shift);
  259.         return $output;
  260. }
  261.  
  262. sub fetch_data ($$)
  263. {
  264.         my $source      = shift;
  265.         my $sourcefile  = shift;
  266.  
  267.         my $return_data = ();
  268.         my $data        = get("http://$source$sourcefile");
  269.  
  270.         # IF the remote data is present, take out the numberic time value and return that
  271.         # ELSE obscure the data, which will revert to 1969/1970 (depending on machinetime).
  272.         if ($data)
  273.         {
  274.                 chomp $data;
  275.  
  276.                 # TODO: Compatibility requirement till mirrors are on 1.6
  277.                 if ($data =~ /^(\d+)/)
  278.                 {
  279.                         return $data;
  280.                 }
  281.                 else
  282.                 {
  283.                         $return_data = $data;
  284.                         $return_data =~ s/\n/\ /;
  285.                         if ($return_data =~ /\S+ \S+ (\d+) \S+/)
  286.                         {
  287.                                 $return_data = $1;
  288.                         }
  289.                         return $return_data;
  290.                 }
  291.         }
  292.  
  293.         else
  294.         {
  295.                 $data = 0;
  296.                 return $data;
  297.         }
  298. }
  299.  
  300. sub process_mirrors (%) {
  301.         my %mirror_time;
  302.         my %mirror_list = @_;
  303.  
  304.         for my $target(sort keys %mirror_list)
  305.         {
  306.                 $mirror_time{$target} = fetch_data($target,$CONFIG{sourcefile});
  307.  
  308.                 # We should probably dont need to use this since it is implied in the routine itself.
  309.                 if(!$mirror_time{$target})
  310.                 {
  311.                         $mirror_time{$target} = 0;
  312.                 }
  313.  
  314.                 # declare our local time before continueing.
  315.                 my $honeynet_ctime      = time();
  316.                 my $mirror_ctime        = $mirror_time{$target};
  317.                 my $mirror_difftime     = $honeynet_ctime - $mirror_time{$target};
  318.  
  319.                 # If the resulting number is less then zero, the remote host is outdated.
  320.                 my $mirror_timediff     = $CONFIG{'timeout'} - $mirror_difftime;
  321.  
  322.                 if ($mirror_timediff gt "0")
  323.                 {
  324.                         if($CONFIG{'verbose'} ne "0")
  325.                         {
  326.                                 $status = "OK";
  327.                                 $timestamp = scalar localtime($mirror_time{$target});
  328.                                 $reporthost = $target;
  329.                                 write;
  330.                         }
  331.                                 next;
  332.                 }
  333.                 else
  334.                 {
  335.                         # The email option is enabled and we found an outdated mirror
  336.                         # jump to the email_report function.
  337.  
  338.                         if($CONFIG{'enable_mail'})
  339.                         {
  340.                                 email_report($target,$mirror_time{$target});
  341.                         }
  342.  
  343.                         if($CONFIG{'verbose'} ne "0")
  344.                         {
  345.                                 $status = "FAIL";
  346.                                 $timestamp = scalar localtime($mirror_time{$target});
  347.  
  348.                                 # Only override if the message has the old timestamp and thus is broken.
  349.                                 if($timestamp =~ /19[6-7]\d/)
  350.                                 {
  351.                                         $timestamp = "Mirrorprobe file problems!";